community.aegirproject.org
Aegir Handbook
Aegir is a set of Drupal modules that helps you maintain your Drupal sites.
What is it?
Aegir is a powerful hosting system that sits alongside a LAMP or LEMP server to create, deploy and manage Drupal sites.
Once Aegir is installed, you can setup a Drupal site in just a few clicks. Aegir creates the web server's site configuration files, the site's database, runs the Drupal installation process and reloads the relevant services, all automatically.
This documentation will help you understand Aegir and how to get it set up and running, as well as how to use it on an ongoing basis.
How does it work?
It does this by providing you with a simple Drupal based hosting frontend or control panel for managing your entire network of websites.
To deploy a new site you simply have to create a new Site node. That site node then provides you with a collection of tasks that can be executed against that site (such as Backup, Restore, Migrate (upgrade), Clone, Disable, Delete, among others).
What is the meaning of Ægir?
In Norse mythology, Ægir was the god of the oceans and if Drupal is a drop of water, Ægir is the deity of large bodies of water.
Another common spelling is "Aegir," which means a tidal wave.
Okay... but how's it pronounced?
Good question! Technically, /ˈaiːjɪr/, but Icelandic phonology is somewhat complicated. The written pronounciation would be something like "ay-eer".
(BTW, most of us at Koumbit pronounce it "eager" in English, and "agir" (to act) in French.)
Where can I learn more?
You're in the right place! Browse the subsections listed in the Notebook sidebar to continue reading our handbook.
Want to extend Aegir or join the development effort? Read our section on Developing for Aegir.
Want to get involved and help improve the project with support, documentation, Q/A, publicity or even just spend some time with our great community? Check out Community Support. Need help? The FAQ is your first point of call.
Levels of access and experience
Every article is tagged with corresponding user level experience or access level required to make use of explained Aegir feature.
Sysadmin level is generally for users with the root access to their server, who are responsible for installation and configuration of entire Aegir system environment.
Hosted level is for users, who are using hosted Aegir version, without access to the server root, with features limited to those allowed by their host, and often with Aegir UI changed/simplified.
Client level is for users who have access to only their sites and not able to manage all Aegir features, accounts or the server/system.
Some topics can include information useful only for Sysadmin level users, while other are useful for all levels. Hosted and Client levels can have further differences. If you are in doubt, please consult your host FAQ pages.
Finally, welcome to our community, and thanks for using Aegir!
Referenced by | Last post | Author |
---|---|---|
reshuffled the handbook | Sat, 11/27/2010 - 01:10 | anarcat |
Introduction to Aegir
If you are new to Aegir you may have heard great things about the project but not know about many of the benefits of using it. Here are some - feel free to add to this list.
Introduction
One of the great architectural features of Drupal is that you can have multiple web sites that share a common platform of modules, themes and other code used to manage content and appearance. If you make a change to the platform (e.g. update a module, change a theme etc.) all sites based on the platform are immediately updated. At the same time, each site has its own database and file system that can be tailored to the specific needs of the site. In addition to the concept of platforms, Drupal lets you create installation profiles. A platform can have one or more profiles which allows you create different sites based on your choice of platform and profile.
Multiple Site Installations
Aegir provides a web interface to define a platform and then create multiple sites based on that platform. Aegir creates the site database, site-specific file folders and the web server files that point browser requests to the right place.
Site Management
Through the Aegir interface you can create backups, restore a specific backup, disable or delete individual sites. Thus Aegir is a very helpful way of tracking and managing lots of sites spread over several platforms. Moreover, you can perform bulk tasks on all sites based on a particular platform.
Site Development
When you are building a platform from scratch or simply extending or upgrading an existing platform, Aegir can significantly speed up development time and minimize impacts to sites in production.
With Aegir you can clone a site to the same or a different platform. So, say you have a number of sites running on a Platform A and you decide to upgrade a platform module but you want to test your sites to make sure they are compatible with the upgrade. Using Aegir you would create another platform, Platform B that is identical except for the module upgrade. Then you can use clone to create a copy of one of your sites from Platform A to Platform B. The database and all your files are copied by Aegir and you can then test your cloned site without worrying that you might impact the existing production site.
Once you are satisfied that your cloned site is working properly you can then migrate one or all of your sites from Platform A to Platform B.
This process can be very useful for building out new platforms, troubleshooting issues etc.
Site Aliasing
Through the Aegir interface you can define aliases for your web sites so that, for example, traffic to http://www.example.com and http://example.com is directed to the same site.
Multiple Servers
The Aegir interface can also be used to manage platforms and web sites on different web servers, which gives you latitude to tailor a hosting architecture to your own needs.
Scheduled Tasks (Cron)
Most Drupal sites rely on recurring scheduled tasks also known as cron jobs to perform operations like checking for updates, cleaning up log files etc. Aegir can be configured to run cron jobs for hosted sites on regular intervals without having to utilitize other tools or configure cron jobs manually on a site-by-site basis.
Operating system support
Aegir is primarily developed on Debian GNU/Linux, but it supports a whole slew of other platforms, Debian packages have been available for that platform for a while and the install instructions have been originally written primarily for Debian.
This page documents Aegir support for various operating system, including specifics on how it is best recommended to install Aegir on those platforms, quirks and problems, packaging issues and coordination.
1. Debian support
1.1. Fully supported
Aegir is fully supported in Debian, with multiple reports of successful installations, both manual and automatic installs and upgrades.
1.2. Packaging complete
Debian packaging is currently done in Koumbit's private repositories, as documented in the install instructions.
Drush is also fully supported in Debian, with regularly Debian packages in unstable, Wheezy, and Squeeze-backports.
Drush make doesn't yet have a Debian package.
To building Debian packages, follow those instructions.
2. Ubuntu support
2.1. Well supported
Because none of the core developpers actually run Ubuntu daily, it is less well supported. But because Ubuntu is actually based on Debian, lots of people are happily running Aegir on Ubuntu without problems.
2.2. Packaging almost complete
- Through the Debian packages, packaging is almost complete. The key piece missing is Drush in Ubuntu Maverick and previous versions. See this bug report for more information.
-
(Ubuntu synced Drush 4.4 in Natty Narwhal 11.04, which should be released by the end of april, but we're still looking for a Ubuntu MOTU to handle the backporting on the Ubuntu side of things.)
-
There are unofficial install scripts also available for Ubuntu: https://github.com/doka/install-aegir-on-ubuntu
3. Arch Linux
3.1. Well supported
Some people definitely run Aegir in Arch Linux, as the manual install process documentation has extensive details of the Arch Linux specific steps.
3.2. No packaging
Unfortunately, no package is available for this platform.
4. CentOS and Redhat-descendants
4.1. Well supported
A few people are running Aegir on RedHat and related operating systems, mostly CentOS. There are a few tricks documented in the manual install process but it's still fairly straightward to install.
4.2. No packaging
Unfortunately, no package is available for this platform, we are looking for volunteers to write a spec file to create RPMs.
5. Gentoo
5.1. Supported
We believe that Aegir works on Gentoo right now, according to this post. Follow the manual install instructions.
5.2. No packaging
No packages are provided on this platform.
6. Mac OS X
- Install using the special instructions for OSX and the manual install process
- Upgrade using the automated script
6.1. Partially supported
OS X can be difficult for web developers, because it doesn't ship with a good Apache/PHP/MySQL stack and the UNIX internals are a bit awkward. Still, Aegir was originally developed on OS X, so it's perfectly possible to use Aegir on OS X.
6.2. No packaging
There are currently no packages for OS X and none planned. Special instructions are provided for installing on OS X but once it's installed, the usage should be similar to other platforms, including for upgrades.
7. Solaris
7.1. Supported
There are at least two Solaris installs out there, and significant work was done to make sure that Aegir was UNIX-agnostic enough to respect Solaris' idiosyncracies. The manual install instructions include some of those exceptions clearly documented and outline. Because the upgrade can be iffy, it is recommended to upgrade using the full manual install process.
7.2. No packaging
No packages are provided on this platform.
8. FreeBSD
- Install using the manual install process and cherry-pick some commands from this HINTS file for prerequisites
- Upgrade using the manual upgrade process
8.1. Supported
We had one report of Aegir being installable on FreeBSD. The pre-requesites have not yet been merged into the main install instructions (see #659024 for followup on that), so you'll need a bit of guesswork.
8.2. No packaging
No packages are provided on this platform, but it should be fairly simple to make a port inspired by the Debian packaging, considering how it currently works.
9. Other platforms
Other platforms are probably supported but those have not been explicitly and clearly documented so they are not yet included here. Feel free to edit this page to add a section for those users!
Project goals
Open
This project is developed using Git. People are free to download the source code for the project (and in fact are required to run it). Git access is available here.
After many discussions with many different developers working in this space, it has become apparent to the developers that many people have developed their own custom set of utilities and scripts to manage many of the issues this project encompasses.
This project aims to be an open framework that will allow us to collaborate on a single set of community developed / maintained utilities, so that the quality and the capabilities will improve for all of us.
Easy
This functions on both a developer and a user level.
For the user, the intent is to make the system as simple to install and operate as possible, which means drastically lowering the requirements for installing and getting up and running with the system. To that end the system has an installation script to automate the installation of Aegir, which has developed a long way from a multi-step install.php wizard and even more complex requirements before then.
The system will auto-detect as many of the settings it requires to run as is possible from within the confines of PHP, and will provide detailed and accurate inline documentation on the server configuration required, based on the configured settings.
This system aims to be self documenting, to aid in troubleshooting, in the event that an error occurs. This documentation is both translatable and extendable when new services are enabled.
For developers we aim to keep the implementation simple, clear and concise, and use as few as possible additional requirements outside of Drupal itself. We aim to provide extensive developer documentation, to lower the barrier for people wanting to integrate with the system.
The system will never require any additional core patches for the sake of base functionality. It is a 'contrib' project.
Basically, the aim is to do everything possible to ensure that the user and developer experience is a positive one.
Useful
The system aims to help maintain many sites, not just when they are initially installed, but over the entire lifetime of the site.
It is designed to install new sites, provisioning all the required services (such as Apache, Mysql and DNS), with the flexibility to provision additional services (such as Mail, LDAP etc.) through optional contributed modules or features.
It provides the ability to revert back to previous versions of your sites, as well as re-deploy existing sites under different domain names (mysite.com copied to dev.mysite.com). These backups will be usable on any system with the provisioning backend installed (such as a developer's machine, or a staging server).
This aspect of the system will also be used to manage upgrades between multiple versions of Drupal, and will allow you to schedule upgrades. It will do its utmost to ensure that your site has been successfully upgraded, and features dependency tracking to keep track of which sites are eligible to be upgraded.
It will provide a mechanism that ensures the cron process is run on all your sites within the specified time frame, and additionally will allow for scheduled backups of all your sites.
Through optional modules it will be able to extract usage statistics from your hosted sites and provide an overview of site usage through the administration interface, it will also be able to keep track of server health and load for each of the configured servers.
These are the stated feature goals, many of these features will not be available in the initial releases, as they are not considered a base requirement.
Secure
The system will make sure that it has the least amount of power available to it to accomplish it's task, and will ensure that all file permissions are set to the strictest possible levels they need to be, and that no sensitive information is web accessible.
None of the functionality or code required to accomplish it's tasks will ever be available in the provisioned sites, and the hosting front end itself will make extensive usage of Drupal's access control requirements to ensure that users will not have access to things they are not meant to see.
to be extended
Distributed
The system functions entirely on the command line, using unix pipes for inter-process communication.
As such, once ssh has been configured for the user the back end script is running as, it will be able to freely communicate with the external server through the only mechanism it knows how (the command line).
As an extension of the backup/restore functionality, you will be able to seamlessly move sites between servers, while providing a redirect to an always accessible site alias for users who have cached dns entries.
You will also be able to change database servers through simply modifying the site record on the user frontend.
to be extended
Diagnostics
The system will provide accurate and complete logs of everything that has occurred on the system, as well as feature complete error reporting and notification of all issues that have occurred. It will first verify that it is able to complete the requested task, and then it will verify that the task has been completed successfully, before completing the task.
When an error occurs, it will rollback to its last working state before exiting, so that failures will not negatively affect sites.
All node types in the front end are revision controlled, to allow for complete history of all objects.
to be extended
Flexible
This system aims to be as flexible as Drupal itself is.
It will provide a complete and useful set of hooks to module developers, for both the front end and the back end, to accomplish whatever tasks they need.
The administration interface will provide complete views support for all data in the site, to allow for custom reporting on your network.
The core node types are all standard Drupal nodes, that can be extended through CCK by the implementor. This could be used for tracking additional client information or additional information on sites.
You will be able to integrate Drupal contributed modules such as Organic Groups or Ecommerce/Ubercart to build support or billing functionality directly into the hosting front end.
The system is completely white boxed, so that it will work with almost any Drupal theme, and as such can have your own corporate identity.
to be extended
Project history
Aegir is a completely new iteration of the original Hostmaster project written by Adrian Rossouw. The original version has been running the hosted service provided at http://bryght.com for nearly 4 years, hosting literally thousands of sites, and has been implemented for several large clients. Originally, the system would only manage a single Drupal instance, on a single web server with a single database server, and as the requirements grew (concurrently running multiple versions of Drupal and then needing to manage several servers), it grew considerably in (unplanned) complexity.
There were also non-ideal implementation and licensing issues, such as the choice of Python and PostgreSQL for the backend and not developing the system as an "invite-only" GPL project, which severely limited the number of developers that were attracted to the project. Because of the system requirements, it was also very difficult for new users to install, and had many unnecessary points of failure.
The Aegir hosting system has been designed with all these issues in mind, and the lessons learned have formed the basis of many of the Aegir Project Goals. The code was committed to Drupal.org in november 2007 and has seen multiple production-ready but still "alpha" releases there in the two following years.
- 0.1 marked the first Drupal 5 and "multi-site" capable version.
- 0.2 marked the first multi-platform version, which supported upgrades between different Drupal versions
- 0.3 marked the port of the frontend to Drupal 6
Then in 2009, a major overhaul of the code was performed to provide "multi-server" support, which led to numerous 0.4-alpha releases (about 20) and, around the end of 2009, required the move to git as a version control system even though Drupal.org was still stuck in CVS-land, which meant Koumbit.org started hosting the git repositories on their servers.
All this led to, about a year and a half later (early 2011), to the 1.0 release of our beloved platform and the Aegir project migrating back to drupal.org at last
Aegir now exceeds the capabilities of the original HostMaster system and is in use by numerous real world projects.
Aegir is now supported by Koumbit.org, Computer Minds and mig5 and is being used to deploy sites in universities, private companies and non-profits around the world.
On January 8, 2014 the 2.0 release ships with significant stability improvements, Drush 5 and 6 support, subdirectory multisite support, improved nginx support, native views and much, much more.
FAQ
- 1. Help!
- 2. Installing Aegir
- 2.1. Can I run Aegir on a shared hosting environment?
- 2.2. Can I run Aegir on Windows?
- 2.3. Can Aegir be installed next to CPanel/Plesk/AlternC/etc?
- 2.4. Where can I get support for Barracuda/Octopus Aegir?
- 2.5. Do I really need to create an aegir user?
- 2.6. Install fails with 'dummy connection failed to fail' on Ubuntu 12.04 LTS!
- 2.7. How can I tell what version of Aegir I have installed?
- 2.8. After installing Aegir I have two servers listed, is this normal?
- 2.9. I followed the installation instructions. What directory holds all of my sites now?
- 2.10. I tried to create a site, but the create task failed and retries don't succeed. How do I delete the partially created site and associated Aegir tasks?
- 3. Upgrading Aegir
- 4. Using Aegir
- 4.1. Cron isn't running (scheduled tasks aren't being executed), what do I do?
- 4.2. How do I get debug/verbose info from Drush when running the hosting commands?
- 4.3. How do I update a site's or platform's modules using Aegir?
- 4.4. Is it best to set up sites under aegir as www.example.com or just as example.com?
- 4.5. How can I setup a site to use https:// under Aegir?
- 4.6. Aegir is not verifying / deleting / migrating my site because of file permission problems.
- 4.7. I can verify a platform but Aegir fails to provision a site on it (often OpenAtrium)
- 4.8. A platform with an installation profile has verified okay, but Aegir doesn't seem to be able to use the profile
- 4.9. I tried to migrate a site, but Aegir incorrectly reported an incompatibility between one or more modules in my source and destination platforms. How do I force Aegir to see that the two platforms are compatible?
- 4.10. What about .htaccess settings?
- 4.11. I'm using a Git repo for a site, but symlinks make crazy things happen!
- 4.12. How can I tell what database name my site is using?
- 4.13. Can I use the subdirectory multisite feature of Drupal in Aegir?
- 4.14. My cache/performance settings in /admin/settings/performance keep resetting themselves!
- 4.15. I get [warn] NameVirtualHost *:80 has no VirtualHosts when I restart Apache
- 4.16. I get 'Dummy connection failed to fail' when doing $stuff
- 4.17. How do I make custom changes to a site's settings.php?
- 4.18. How to migrate site via drush
1. Help!
1.1. Where can I get help?
From the Aegir Handbook (which should be the first place you look) :
The Issue Queue for all the projects that make up Aegir is here: http://is.gd/ek2Dl
This is the first place to look - chances are someone's had the same problem as you before, and you'll get the quickest answers by searching for what they discovered.
Before asking a question or reporting an issue, you should read the bug reporting guidelines.
You can join the #aegir channel at Freenode on IRC. (IRC instructions for drupal users are at: http://drupal.org/irc). It's a friendly community, but a small one, so ask nicely and be patient!
If you find a solution to a problem that isn't in the documentation or in this FAQ, be the first to login and edit these pages to bring them up to date! The rest of the community thrives on your helpful information!
1.2. Can I ask you a question?
Of course you can! Don't ask to ask, just ask! It is polite to look if the question has already been answered in the documentation, the issue queue or here, but if not, please do ask around! If you ask a question and we have a nice answer, you can even add it to the FAQ here so that others can have your valuable answer.
1.3. Who are the Aegir developers?
The Aegir core developers are listed on the Aegir maintainers page.
1.4. I'd like to help out / become a developer!
We love people who want to help and get involved! We even made a page for you to read.
2. Installing Aegir
2.1. Can I run Aegir on a shared hosting environment?
Shared hosting will not give you enough permissions to install new sites. Toughen up, read some Linux beginners tutorials and buy a cheap VPS which will do nicely.
2.2. Can I run Aegir on Windows?
To install Aegir you need a unix based operating system (for details see: http://community.aegirproject.org/handbook/operating-system-support). Aegir will not work on Windows and there are no plans to add support for it.
2.3. Can Aegir be installed next to CPanel/Plesk/AlternC/etc?
No, or rather "it's not supported". See http://drupal.org/node/587554
2.4. Where can I get support for Barracuda/Octopus Aegir?
Barracuda/Octopus are custom scripts for Aegir that install or set up various things that Aegir doesn't do out of the box. This is a custom solution tailored to Nginx systems, provided by Omega8.cc and is not officially supported by the core Aegir project or team. For help or more information, please visit BOA Group, Barracuda and Octopus project pages / issue queue on drupal.org.
2.5. Do I really need to create an aegir user?
If you have a UNIX system, you already have a user. It just needs to not be root for safety reasons. But you don't really have to create a user.
2.6. Install fails with 'dummy connection failed to fail' on Ubuntu 12.04 LTS!
Ubuntu 12.04, by default, installs an insecure MySQL server configuration that allows anonymous users to login without a password. This must be rectified by running 'mysql_secure_installation' before attempting to apt-get install Aegir.
Please see this step in the Install Documentation and this other FAQ item.
2.7. How can I tell what version of Aegir I have installed?
This works for version 1.1 and above . . . maybe earlier too . . . .
grep '[hostmaster][download][tag]' /var/aegir/.drush/provision/aegir.make
Also, if you have drush installed someplace other than in /var/aegir/.drush, then you should grep the aegir.make file in the directory tree where you have drush installed. For example, on Ubuntu, apt-get may install drush in /usr/share/drush/. So, the file to be grep'd would be /usr/share/drush/commands/provision/aegir.make.
If you're still unsure or this didn't work, contact a developer on IRC and we'll be able to tell you after you answer a few questions.
2.8. After installing Aegir I have two servers listed, is this normal?
Yes. By default, after installing Aegir, a server node is created that represents your webserver. This also generates the 'server_master' Drush alias on your system. The second server node is called 'localhost' and has your database service attached to it, because by default MySQL listens on localhost and not your webserver's IP (or on all interfaces). This corresponds to the 'server_localhost' Drush alias on your system.
This is normally fine: you want your websites to be serve-able via your web server's IP / all interfaces, and your database server to only listen on localhost.
If you end up creating remote webservers, you'll not be able to have them use databases on this Aegir server is all.
If you know you want to have remote webservers communicate back to the Aegir server as the 'remote database server', you'll need to attach the database service type to the non-localhost server node, comment out bind-address in /etc/mysql/my.cnf and perform the necessary firewall changes to allow remote access to TCP port 3306 on the Aegir server.
But generally its recommended to create remote database servers too (or combine web + db on the one remote server), so this is generally a non-issue, rather than use the original Aegir server as a 'remote' database server for remote web servers.
2.9. I followed the installation instructions. What directory holds all of my sites now?
For a full overview of a typical Aegir filesystem structure, read this.
2.10. I tried to create a site, but the create task failed and retries don't succeed. How do I delete the partially created site and associated Aegir tasks?
See the section 'Manually deleting the site' from the documentation
3. Upgrading Aegir
3.1. What is a patch or 'hotfix', and how can I apply it to fix my site if I can't / don't want to upgrade to HEAD?
Sometimes releases are made and bugs are discovered afterward. We commit fixes for such bugs to HEAD, but HEAD is often volatile and you don't want to use a non-stable release in your environment. Fortunately we do (or should: if it's critical, please ask for a patch otherwise) provide a 'hotfix' or patch for users. A patch means that you can 'patch' your Aegir system to fix the bug without running the risks of using HEAD.
To patch an Aegir component, use the following example, in which we fix the Provision module with an imaginary patch. The same or similar steps apply for Hosting or Hostmaster (though you shouldn't need to patch Hostmaster, you only use it once to install!)
Run the steps as the aegir user.
1) Backup /var/aegir/.drush/provision to somewhere else for safety, such as /tmp/provision
2) Download the patch given in the ticket, to your server. Use wget or a similar useful tool for downloading it over the commandline. You can download the file to anywhere.
3) edit your aegir user's crontab (crontab -e) and comment out the dispatch entry (place a '#' in front of the hosting dispatch cron entry). This is a precautionary step in case you patch the module, something goes wrong and it breaks the module. The hosting dispatch cron might come around and try and run the queue dispatcher and break/complain due to a broken component. If you know what you are doing and are fast enough to fix it, you may skip this.
4) cd /var/aegir/.drush/provision/
5) patch -p0 < /path/to/the/patch/file
6) crontab -e and uncomment the dispatch cron task
If the patch doesn't apply or prompts you to enter the filename to patch, try running patch -p1 instead of -p0 above.
If it still asks for clarification, verify that you are in the top-level directory of the component (in this case, we are inside /var/aegir/.drush/provision)
If it still doesn't work, it might be an unclean patch, and you should make a request in the relevant ticket to be given a cleaner patch to apply.
For more information on patches, please read the Drupal documentation.
3.2. I upgraded Aegir by mistake, can I downgrade it?
Short answer: no, that would be crazy.
Long answer: well... maybe. Actually, yes. But it could destroy your house, your cat, your life insurance and certainly give you headaches. We will tell you anyways because we're nice.
This assumes you have a backup from before the upgrade. We will be deploying that backup, so understand that you will loose any changes done on the frontend since the upgrade was performed (or more precisely, since that backup was done). That's the way Drupal works, you can't downgrade Drupal, and we're just cheating by restoring a previous backup.
So first, delete that frontend site:
drush @hostmaster provision-delete
This is not absolutely mandatory, but you should make a backup, and deleting the site does that, plus it's important to cleanup after ourselves. We can use the backup to restore to that upgraded version if we give up on the downgrade.
Then remove the Debian packages (if you installed through those, if not, you're on your own: but you need to at least remove the provision code):
apt-get purge aegir2 aegir2-provision aegir2-hostmaster
This may also be:
apt-get purge aegir2 aegir-provision2 aegir-hostmaster2
Depending on the release candidate you installed.
Next, you install the old provision, but only that - don't go around installing the frontend just yet because you will have an empty site to overwrite.
apt-get install aegir-provision
Then you need to edit the @hostmaster
alias to point to the old platform, which supposedly exists (if not, you're in trouble: create it by hand or something).
(Alternatively, you could install aegir-hostmaster, which would create you the alias, but this could fail exactly because the alias already exists. Just edit the alias already.)
So you need to change three parameters:
'platform' => '@platform_hostmaster6x1x', 'root' => '/var/aegir/hostmaster-6.x-1.x', 'site_path' => '/var/aegir/hostmaster-6.x-1.x/sites/aegirphp52.koumbit.net',
Then you can deploy your backup on that platform:
drush @hostmaster provision-deploy $PWD/backups/aegirphp52.koumbit.net-20131017.173034.tar.gz
From there the frontend should be back. Maybe. Probably. Well, the purge earlier may have killed the aegir.conf symlink, so if you want to test the frontend right now, restore that. Otherwise, be bold and upgrade to the latest 1.x release using the debian packages, which should restore everything you need:
apt-get install aegir
This should perform an upgrade from your older 1.x release to (for example here) 1.11.
If it doesn't work, sorry, i told you it could break. :)
4. Using Aegir
4.1. Cron isn't running (scheduled tasks aren't being executed), what do I do?
If you're using Aegir 2.x ensure the "Task queue" feature is activated. To do so go to /admin/hosting/queues check "Task queue". Click on "Save changes" button. With default settings the item(s) in the queue will be process in roughly 1 min.
Below are other options to resolve that issue
At the shell prompt as the aegir user, type
crontab -e
a) If you get an error that the command is not found, then you need to install cron on your server (some basic images don't include it - Linode's Ubuntu 9.04 image is one):
sudo apt-get install cron
/var/aegir/drush/drush.php @hostmaster hosting-setup
b) You should see the default entry:
*/1 * * * * (php '/<path to drush>/drush.php' @hostmaster hosting-dispatch
But if this is still not working it may well be a permissions issue. An example error :
Forking : (php /<path to drush>/drush.php --quiet @hostmaster 'hosting' 'task' '266' --backend &) > /dev/null [0.432 sec] [notice]
sh: /dev/null: Permission denied
This means the system doesn't have permissions to access /dev/null. To resolve it, give the system permissions to write to /dev/null (it should have this anyway), by executing:
sudo chmod a+w /dev/null
c) If this still hasn't worked, have you moved the location of drush? If so - see http://drupal.org/node/540152
d) If you are a *BSD user (verified on freebsd at least)
You need to add the line :
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
at the TOP of your cron file.
Simply log in as the aegir user and type crontab -e and insert the above line on top.
Thanks goes to mauror for http://drupal.org/node/323732.
e) When cron for your sites stops working, it is possible that for some reason (system overload, broken site, timeout etc), Aegir failed to release a sites cron semaphore. To release it, use this simple recipe:
$ su -s /bin/bash - aegir
$ cd /path/to/hostmaster/sites/domain
$ drush vdel hosting_queue_cron_running -y
4.2. How do I get debug/verbose info from Drush when running the hosting commands?
Add these to the drush call :
--debug
If you're having specific problems when using the frontend, you can easily run the hosting commands from the command line by:
- Click the view button and then copy the Drush command which was executed at the top of the log;
- While logged in as the aegir user, paste the command in at the terminal and replace --backend with --debug at the end of the commandline.
4.3. How do I update a site's or platform's modules using Aegir?
Aegir isn't a package management GUI for installing modules or themes on your site.
It's designed to help manage your site between builds. To upgrade sites, you should be using the Migrate tool to move your site between two platforms, which invokes drush updatedb.
If you aren't ready for this paradigm, and are frustrated that Aegir isn't some GUI for wildly updating or installing new modules with no regards for rollback or dealing with when it will inevitably break your site, Aegir is probably not for you.
4.4. Is it best to set up sites under aegir as www.example.com or just as example.com?
Aegir supports either method and can even automatically create whichever one you didn't use as the main site name. Read more about Aliases and this feature in our documentation.
4.5. How can I setup a site to use https:// under Aegir?
Aegir ships with an experimental SSL feature, but currently doesn't provide any means of managing SSL certificates.
You can follow the ongoing SSL work here.
In the meantime, to set up SSL with basic self-signed certs, follow our documentation.
4.6. Aegir is not verifying / deleting / migrating my site because of file permission problems.
Aegir releases prior to 0.3 RC3 encountered an error when the web server has uploaded files and made new directory in the sites/$url/files directory. A prevalent example of this occurring is with the color module and when imagecache is used (such as in atrium).
This bug (see: #203204 ) was introduced with Drupal 6, and has since been fixed with Drupal 7. The patch has however not been backported yet, and the patch won't fix existing files that were created incorrectly.
In Aegir 0.3 RC3 we introduced a work around which sets the umask for the site, which forces new files and directories to be the correct permissions, but this will only take effect after the site has been succesfully verified. To enforce the correct file ownership and permissions on the files, run the following command on each of your Drupal 6 platforms :
sudo chown -R $aegir_user $platform/sites/*/files
This will allow the verify task to correctly set the permissions and succesfully modify/remove the files.
4.7. I can verify a platform but Aegir fails to provision a site on it (often OpenAtrium)
A premature failure of a site install with an unhelpful message such as 'The external command could not be executed due to an application error' often, but not always, means your php-cli's memory_limit is set too low.
To confirm that this is an issue, try running drush @example.org provision-verify --debug from the command line to find out what caused the problem. Remember to replace 'example.org' with the URL of the site you are trying to verify.
Raise your memory_limit in your php cli php.ini file to something like 128MB, or maybe 64MB if that's too much. On Debian-based distributions this is /etc/php5/cli/php.ini. Apache does not need a restart for this: installations fail because the command is being executed via drush using php cli.
This is a common issue for OpenAtrium as it requires more memory to run its install profile as it has a lot more to do than a regular Drupal install.
4.8. A platform with an installation profile has verified okay, but Aegir doesn't seem to be able to use the profile
Your platform has been setup and verified okay, but when you go to create a new site Aegir keeps giving an error message about the profile, or the list doesn't offer you the profile you're expecting. This could be a namespace issue. Modules, themes and profiles in Drupal all have a machine name - which is essentially their folder name. This is also used to prefix their functions etc, so that there are no clashes. What isn't often recognised or well documented is that Drupal has a single namespace for all items - so your themes, modules and install profiles can't have the same name as each other. Often an installation profile will be created called 'OpenPotato' and the theme for it will also be called 'OpenPotato'. This makes Drupal unhappy, and when Drupal's unhappy Aegir's unhappy. We don't want that, but it isn't Aegir's fault, it's the developer's limited or lack of understanding of this Drupal caveat. The easiest thing is to name your installation profile directory 'OpenPotato_install' and name the profile file 'OpenPotato_install.profile'. You'll also need to rename the functions inside this file. Then everything should work just fine.
4.9. I tried to migrate a site, but Aegir incorrectly reported an incompatibility between one or more modules in my source and destination platforms. How do I force Aegir to see that the two platforms are compatible?
This problem can arise for at least two different reasons. First, you may have updated a module in a platform that you imported and forgotten to run update.php before you imported the site into Aegir. Second, you may have updated a module in a platform and not re-verified the platform in Aegir. Be sure that you have run update.php on a drupal platform that you will be importing into Aegir before you import. Also, if you make any change to the modules in an Aegir Platform, re-verify that platform and all sites using that platform. (At least, experience suggests that doing these things will allow the migration to succeed.
4.10. What about .htaccess settings?
As of recent 0.4 alpha releases, Aegir now reads in the contents of a platform's .htaccess file when generating the Apache configurations. You can safely make additions/changes to the platform .htaccess and expect the relevant site to respect those changes.
4.11. I'm using a Git repo for a site, but symlinks make crazy things happen!
If you're symlinking an entire /sites/ folder, Aegir may not acknowledge the site, as this will cause problems when cloning/migrating a site.
If you are using Git to version control your themes, custom modules, etc, and symlinking these into your sites folder, you will find that cloning a site will clone those symlinks (by virtue of copying the sites folder). This is probably not what you want to happen!
Themes and custom modules are better off being checked out to or symlinked into the install profile folder. If your site's install profile is 'foobar' (there's an 'install_profile' variable in the variables table), Drupal core understands to look inside the /var/aegir/yourplatform/profiles/foobar/ folder for modules and themes directories.
The files directory for uploads (which I don't think can be efficiently versioned anyway in most cases) and the settings.php file will be all that's left in your sites folder, and Aegir will happily move that all around as you clone or migrate your site between platforms.
Storing your site's dir in git is an archaic and inefficient method of developing sites. It's 2010 and there are more elegant ways of doing things thanks to Drush Make and Aegir. You might like to look at mig5's excellent article where he provides some great tips on how to manage Drupal deployments & workflows with version control, drush_make, and Aegir.
4.12. How can I tell what database name my site is using?
Check the site's vhost configuration file in /var/aegir/config/server_$whatever/apache/vhost.d/. The db name (and user/password) are stored as 'SetEnv' parameters to Apache, allowing us to cloak database credentials in the settings.php so that they are protected in a multisite environment from prying eyes (admins with PHP filter permissions).
You can also use 'drush st' on your site to get a status report containing the database name, user, password, among other things.
In modern releases (0.4-alpha9 or so and above), the site database names have switched from site_$nid style names to more properly reflect the site name/URL itself.
4.13. Can I use the subdirectory multisite feature of Drupal in Aegir?
Also known as "http://example.com/site1" and "http://example.com/site2" are different sites.
Short answer: not yet - at least not natively. (see the 'aegir_subfolders' experimental extension)
Long answer: this is currently not supported, because multiple sites usually end up sharing the same virtual host entry in the webserver and you will be required to upgrade/migrate all of the sites simultaneously. That is because /site1 is on the same drupal root as /site2. Another problem is that they can't be on separate servers...
To fix this, you would need to decouple virtual hosts and sites in Aegir, which is a fairly deeply rooted assumption. It would be possible to use Aliases to work around that and have the sites be in different document roots.
But even if you actually coerce Aegir into having multiple sites in the same domain, you get into crazy namespace issues. Do you want to allow the site on example.com to completely break sites example.com/site1 or example.com/site2? If so, who can? The same client? The same user? If not, you need to implement checks so there's no overlap...
Tools shouldn't dictate policy, people should. Therefore, this should probably be allowed, from a strictly technical perspective.
The thing is: domain names are cheap, contrary to common opinion. You only have to get one domain and you can have as many sub-domains as you want. Hell, you can even have sub sub domains and have your whole domain tree down there, you're free to go. Providers often give out a free DNS namespace to their customers based on their username (full disclosure: Koumbit does :P).
The other thing is: /site1 is exactly what drupal does: ot dispatches menus callbacks based those GET URLs. So trying to make Aegir (or Apache + Drupal, rather) do that is rather backward. On the other hand, Drupal is notoriously bad (if you make exception of the Domain module) at dispatching requests based on domains, and that's where Apache, DNS, Aegir and all their happy friends gets in.
In short, it's been very low priority for the use cases of the development team so far. If developers step up or if funding is provided, this could very well be part of a future release. See this feature request for followup: http://drupal.org/node/705026
Also see an extension for Aegir, written by mig5, called 'aegir_subfolders', which works (mostly)
4.14. My cache/performance settings in /admin/settings/performance keep resetting themselves!
Some values such as $conf['cache']
are hardcoded in the site's settings.php, which Aegir generates. This prevents some settings from being changed in the admin interface.
You can change the value of cache, and other variables, by creating a brand new file called 'local.settings.php' inside your site directory (e.g, alongside the settings.php). Put any value you want in it, e.g:
<?php
$conf['cache'] = 3;
..leave off the closing ?>
tag, and this setting will now override all others. Aegir will not edit or remove this file.
4.15. I get [warn] NameVirtualHost *:80 has no VirtualHosts when I restart Apache
This is a harmless warning that Apache throws when you have duplicate 'NameVirtualHost *:80' entries in the Apache config somewhere. It doesn't cause any problem.
Aegir inserts this config line into an Aegir-managed apache config file just to make sure it's there at least once. If it's annoying you and you want to remove this warning, you need to remove the IP address from the server node and verify the server again. Note that this may fail in 2.x if the IP address is used by an SSL certificate yet the SSL certificate is unused. This shouldn't generally happen unless you are running a pre-2.0 release candidate, see #2159265 for details and [/discuss/how-delete-unused-ssl-certificate#comment-2163](this workaround).
4.16. I get 'Dummy connection failed to fail' when doing $stuff
Aegir attempts to make a connection to your database server that is deliberately designed to fail. The intention of this, is to get the 'remote host' of your webserver as seen from the perspective of the database server. This is used to generate appropriate GRANT statements so that your Aegir-managed sites can connect to their appropriate databases.
If you get the message 'Dummy connection failed to fail', this means that Aegir was able to login to your mysql server using username 'intntnllyInvalid' with a null password. This is a security problem on your behalf!
You should secure your MySQL service by running 'mysql_secure_installation' or manually by removing users in your mysql database's User table that don't have a valid password. See http://dev.mysql.com/doc/refman/5.1/en/default-privileges.html for more information.
Alternatively, if your MySQL server is sufficiently secured, there's always a possibility that your MySQL server responded with something Aegir didn't understand. Aegir expects either of these error messages:
Access denied for user(.... etc)
Host (...) is not allowed to connect to (....etc).
If your MySQL server responded with something else, but is secure, please open a bug report.
4.17. How do I make custom changes to a site's settings.php?
Aegir re-generates the settings.php for a site on certain tasks (Verify, Migrate, Backup, Restore, etc). For this reason it is not wise to put custom settings in the settings.php as you will probably lose them.
Instead, create a file 'local.settings.php' alongside your site's settings.php and put your custom changes in there.
If you want to make a change and have it apply to all sites immediately, put that change in /var/aegir/config/includes/global.inc.
Neither of these files will be overwritten by Aegir on any task. thus preserving your changes.
4.18. How to migrate site via drush
drush @www.site.alias provision-migrate '@platform_new_platform_alias'
Then, to update the frontend to reflect the changes:
drush @hostmaster hosting-import '@www.site.alias'
Ubercart Integration FAQ
These are frequently asked questions about Aegir Ubercart Integration. For basic install and usage instructions please consult the guide.
Contents
Does this module create a site install task after product is purchased?
There may be some adjustment to UI needed to make this clearer, but the short answer is "Yes".
This is done conditionally: The module adds a checkbox saying "Create my site later" to the purchase form displayed on a product, if that product has the site feature. If that checbox is checked, then we go right to payment completion, and the client will create their site later using the regular aegir interface.
If that checkbox is not checked, we throw the user into a site creation form, store the data they input, and then use it on payment completion to create the site.
Regardless of whether or not they enter their site info during ordering, their client's quota is incremented.
In the case of a product kit with multiple sites included, only the first site can be created in this manner.
Case Studies
Case studies are intended to help explain the utility of Aegir by showcasing its use in some real-world example.
- Pagebuild Case Study
- Pagebuild allows non technical users to build websites very quickly and easily. It was built on top of Drupal, with the idea behind it being to provide an editing interface that is so simple, even an adult could use it. This is a explanation of what we did and how we went about doing it (and why).
Wikipedia Article
Ref.: http://en.wikipedia.org/wiki/Aegir_Hosting_System
Feel free to enhance and improve!
Getting Help
Need help with Aegir? Documentation just not covering it?
We recommend you run through the following in this order:
- Read the FAQ to check that your problem isn't answered there or that there isn't a workaround
- Review these tips on configuring Aegir, Drush, Apache and more, and avoid common mistakes.
- Search this site to see if your problem or question has already been addressed.
- Look at the current issues for all of the Aegir-related issue queues to see if the bug has already been reported.
- Ask the community for help (but see below for good hints on how to make this work better!)
- If you think you have found a a bug report or support request, head over to the bug reporting guidelines
- If you've exhausted all these avenues and still haven't been able to resolve your issue, or otherwise want some professional assistance, you may want to refer to our Aegir Service Provider Directory
Asking for help
If you need help from the community, you're going to have a lot more success by helping the community help you. Just saying "it doesn't work" or "it's broken" will not help us help you. When you ask for help or report a bug, please include the following information:
- if the install failed, try the install again with the
--debug
flag on thehostmaster-install
command, or with theDEBUG=yes
environment set before running apt-get install - if a task failed, include a copy-paste of the complete task log, properly formatted (between
<code>
tags) - in any case, explain what you are trying to do clearly, what are the expected results and the actual results and how to reproduce your issue
- give as much details as you can: the version of your operating system, the version of Aegir, drush and Drush make you are using, the version of Apache or Nginx you are running, which servers or platforms you are using, etc
For more information about how to ask smart questions, see How To Ask Questions The Smart Way by Eric S. Raymond.
Finally, if you're confused or don't know where to go, you can always come to us! To talk with the community and the developers directly, you can join IRC, discuss site or ask on the mailing lists. See the Community and Support page for more information.
Troubleshooting Aegir
Aegir undergoes rigorous testing, but no actively-developed software ever escapes bugs.
If you feel you have encountered a bug, the FAQ is the first place to check if there's a simple solution.
Common installation and configuration errors
Aegir is a more complex product than your average Drupal site. Something that has far-reaching control over bits of your server and your sites results in more bits and pieces working together, and inevitably it becomes possible to misconfigure, break, or miss setting up any one of these components.
The installation and upgrade procedure has evolved from these pains to becoming something almost purely automated. Nonetheless, making a mistake during the installation of Aegir or Drush is still easy to do, and these tips identify the most common mistakes.
If Something Goes Wrong
If an installation fails partway, particularly while using the install.sh script, you may not be able to run it successfully until you've rolled back any changes that the script has made. Delete the contents of the /var/aegir directory (not the directory itself) making sure to remove any hidden directories like /var/aegir/.drush. Also, drop any new MySQL database (which will be named according to the hostmaster domain name).
The Aegir User
Make sure you have an aegir user (who we call the Aegir User) and that the Aegir User is properly configured on your server.
Typing 'id aegir' on your system should print something like this as output.
aegir:~# id aegir
uid=104(aegir) gid=108(aegir) groups=108(aegir),33(www-data)
If you don't want to use a user called 'aegir' for the installation, that's possible (albeit not recommended). Be cautious about using a different username, and do not use an existing user or a user that will have any other role on the server. The Aegir User has certain privileges and should only be used for Aegir activities.
Aegir User's home directory
Verify the Aegir install directory is the user's home. In a standard installation, the Aegir directory is /var/aegir.
Be cautious about installing Aegir in a different directory - it is easy to get confused since nearly all the documentation tries to be consistent by assuming you have used /var/aegir and you'll otherwise constantly have to translate this mentally when trying to learn.
Aegir User as script user
Make sure that the 'Server' node in your Aegir system (this is usually the node located at /node/2) lists the 'Script User' as being your Aegir User. You can also find this node by clicking on the Servers menu tab and then clicking on the server URI shown in the list that appears of Aegir-managed servers. In a basic installation, there is only one server.
Do not run as root
Do not run aegir as root! It's an unsafe and unsupported configuration that is known to cause problems.
Drush
A standard Aegir installation will check that Drush is functioning properly, or if it doesn't find Drush at all, will install it for you.
Always run drush as your aegir user, and never as "root".
Provision
Make sure that the Provision module is in the aegir/.drush/provision/ directory. A standard Aegir installation will install this for you.
MySQL
Aegir uses the 'root' user of the MySQL server to create and manage databases. If MySQL is running on the same server, Aegir will still attempt to connect to the database as if remotely. If the MySQL server is security hardened to prevent this type of access, this would result in MySQL errors during Aegir installation.
To see the root users. Connect to MySQL database and run the query
SELECT user, host, password FROM mysql.user WHERE user = 'root';
There should be separate records for the host localhost
and hostname
, and the password hashes should be the same to avoid confusion. If the password hashes are different, which is possible if the root user password has been updated with mysqladmin, you can make the passwords consistent by running:
UPDATE user SET Password=PASSWORD('new_password') WHERE host='hostname' AND user='root';
FLUSH PRIVILEGES;
Apache
Make sure the Aegir User can restart Apache without a password and that the corresponding command is defined in your Aegir 'Server' node.
As your Aegir User, run the command shown in the "Restart command" field in your Aegir 'Server' node to ensure that your Aegir User can in fact restart Apache (if not, see the Sudo information below).
On Debian/Ubuntu, the command sudo /usr/sbin/apache2ctl graceful. The command should execute and return without asking for a password when run as the Aegir User.
Make sure Apache can parse its configuration without syntax errors. On Ubuntu, sudo apache2ctl configtest should report "Syntax OK".
Sudo
Check that the aegir user is in your /etc/sudoers file with the permission to restart Apache. Instructions and an example for doing this are contained in the INSTALL.txt.
On a Linux system, check that /etc/sudoers is chmod 440 (i.e., read only for the file owner and group, which is usually root)
Attempt to run the task queue manually from the command line
php /var/aegir/drush/drush.php @hostmaster hosting-tasks --debug
Run a specific task in the queue from the command line
php /var/aegir/drush/drush.php @hostmaster hosting-task <nid of the task> --debug
Run the 'Cron' queue manually from the command line
php /var/aegir/drush/drush.php @hostmaster hosting-cron --debug
Contact the community
Want to talk with the Aegir community? There are many channels with various virtues.
Discussion Forum
The community site has a discussion area that anyone can use to talk about Aegir with others or seek support.
IRC
You can join the #aegir channel at Freenode on IRC. There's a loyal community of Aegir developers and users that frequent this channel. Much development discussion also takes place here, and support questions are welcomed.
The core developers also are reachable in #aegir on IRC.
Remember that Aegir, like Drupal, transcends timezones, and even the developers are many hours apart from each other. Sometimes it's quiet, and sometimes it's very busy. Don't repeat yourself: someone will respond eventually if not immediately.
New to IRC? IRC instructions for Drupal users are at: http://drupal.org/irc.
Mailing Lists
There are mailing lists that can be used for support, general discussion and development or announcements.
- announce - the announce mailing list is used to announce new releases or major news
- users - the user mailing list is used for general discussions and support for Aegir users
- core - the core mailing list is used for internal discussions in the core team
Events
Aegir is often present at many of the popular DrupalCamps and DrupalCons, either as a formal session or in smaller BOF sessions.
We encourage users who are confident in using Aegir to demonstrate it at any relevant events they attend, and to post slides/screencasts so that others can benefit.
Blogs
As well as the News/Blog area here, the developers also often blog about new Aegir features or share handy tips for the community. Here are two blogs to stay up to date with:
Bug reporting guidelines
Reporting bugs is a great way to contribute to Aegir and it helps all of us (developers and users both) shape Aegir into the fantastic Drupal management system we want and need.
Before reporting bugs
Mis-reporting bugs that are due to not following instructions or skipping steps, not providing enough information or duplicating reports, however, can be a distraction to the development team and waste precious resources. So, help out by following these guidelines for reporting bugs.
Before you report a bug, make sure you have done things right and are sure it's a bug, by reviewing the Getting help section, if only to include the right information in your bug report.
Where to report bugs
Somehow a bug or two may have slipped into Aegir. Eek! So it's really a bug? A new bug? To report it, file an issue on Drupal.org:
- Hosting (if the issue is in the frontend)
- Provision (if the issue is in the backend) or
- Hostmaster (if it is in the installer, a documentation bug, or if you don't know where the issue should go).
- Eldir (if it's an issue with the default Eldir theme that ships with Aegir.)
Feature Requests
Feature requests are, of course, always welcome in these queues as well. To get an idea of where Aegir is headed with new features, you should first consult the Aegir Roadmap and Project Goals - we may be already on it!
How to help?
Testers
- download new release
- report bugs
- test existing issues and patches
- use it! test everything!
Documentors
Contribute to this handbook and the rest of the community site.
Contributing can mean adding new documentation, or reviewing existing docs to ensure that they haven't become out of date, incorrect or been defaced.
Promotion
Blog about Aegir and put your aegir term feed in our feed aggregator
You can also make or collate screencasts and post them here for others.
Developers
Want to become an Aegir developer?
Submit good patches and review existing ones and we will love you and give you commit access.
Donate
See the donation page.
Donate to the Aegir project
We are happy to take donations from users to further development and keep our community site, mailing lists, and Jenkins CI server running. Note that you can also talk with existing service providers if you have the budget to further specific functionalities, but otherwise a small (or big!) donation is always appreciated and will make sure we have funds to operate the infrastructure for the project or maybe even pay people for development work.
Help us work on one of these goals:
- The next 7.x-3.x maintenance release
- The next 6.x-2.x maintenance release
- Hosting Drupal 8 sites
- Extending Aegir with new features
You can make this happen.
Financial support can greatly accelerate our development efforts. The core team needs your help to keep this going.
You can make this happen! Can you spare $5?
Gratipay (formerly Gittip) has a team for Aegir
The small core team that works on Aegir does this mostly because they use it themselves. That however does not cover all the time we work on Aegir.
Our passion to produce a good product, and love for Drupal, drives us to keep working on this, but there is a risk of developer burnout.
For example: supporting older versions and making upgrade paths user-friendly is a lot of work. While the core team has only limited personal need for this.
Donations validate more time being spent on this.
Please contact one of the service providers if you're looking for paid support.
Aegir Service Providers
Here is a list of consulting and hosting firms offering paid support for the Aegir Hosting System:
- Koumbit Networks
- Koumbit's team includes several of Aegir's core maintainers and contributors; including, among others, Anarcat, the project lead, as well as developers of several contributed projects. Koumbit manages one of the largest Aegir deployments, hosting hundreds of custom developed websites, as well as being one of the first providers to automate site ordering and provisioning. Koumbit is a dynamic team of designers, developers and sysadmins, offering support and consulting services from front-end development (design, theming and UI), through e-commerce, complex Drupal site builds and data migrations, to hosting and systems administration. This diversity has allowed Koumbit to develop deep expertise in distributed development best-practices. Koumbit has recently launched AegirVPS services, offering full root/SSH/SFTP access to robust virtual servers with Aegir pre-installed, maintained and supported.
- Omega8.cc
- Omega8.cc offers dedicated Aegir hosting service since July 2009, using custom developed, high performance cloud servers configuration. Hosted options include SSH and Drush access, with simplified Aegir interface. Omega8.cc supports also remote Aegir installations and maintenance on Client's servers. Simplified and optimized for single server installations system configuration is open-sourced since 2009 and available as a dual-core Barracuda and Octopus installer.
- Praxis Labs Coop
-
We are a worker's cooperative based in Montreal. We share an interest in web technologies and direct democracy. We've all been working with Drupal and Aegir for many years. One of our founding members (Christopher "ergonlogic" Gervais) is in fact a lead developer of the Aegir project.
We provide Aegir, Drupal and Open Atrium hosting, development, support & consulting.
Praxis is a proud partner of Koumbit networks.
- ThinkDrop Consulting
- ThinkDrop provides Infrastructure and Architecture consulting for Aegir and Drupal. We are the creators of (DevShop)[http://drupal.org/project/devshop], the first open source Drupal Environment Manager, built on Aegir. We have also created a number of other Aegir addons like Aegir Drush Aliases and the Hosting & Provision Solr modules. The ThinkDrop team is Jon Pugh, Kevin LaPalme, and Jacinto Capote
- Initfour websolutions
- Initfour provides Infrastructure and Architecture consulting for Aegir and Drupal. Internally Aegir is used for hosting managed Drupal sites. Fixing what needs to be fixed, Initfour has worked in many corners of the Drupal/Aegir universe.
- On Drupal.org: helmo
User Manual
This section of the handbook describe how to use Aegir in the day-to-day management of sites and platforms.
Getting to know the interface
So you've installed Aegir, arrived at the home page of your installation and are waiting for the magic to begin. On this page we'll introduce you to Aegir and how easy it is to use.
Get to know the interface
Home
Once you've successfully completed an Aegir installation, you'll be presented with the homepage, titled 'Home'.
In a standard installation there won't be any sites listed here yet, other than the main Aegir site itself, which gets imported into the system during the installation. This is normal - Aegir even recognises itself as a Drupal site on your server, and to a limited degree is capable of 'managing' itself. Skynet is here!
Admin menu
At the very top of the screen you'll see the Admin menu, which you may recognise as a popular contrib module installed on many other sites, perhaps even some of your own.
This gives you access to all the normal Drupal administrative functionality, which is not normally required in everyday Aegir user, but it also does provide you access to the 'Hosting' administrative settings where you can enable features that ship with Aegir and make other configuration changes.
The Eldir theme
The Aegir project ships with a default theme called 'Eldir' which is the classic navy blue, simplistic functional interface consistent with the overall Aegir brand. Eldir has been specifically designed for Aegir - nonetheless it is a Drupal theme like any other.
Blocks
Eldir has a main content section and a right sidebar. In the sidebar, several blocks are enabled by default.
Queues
The first is 'Queues'. Queues are Aegir's method of creating 'tasks' and putting them into a pool for the backend system to execute.
Two types of Queue exist in Aegir, though only one is enabled by default. These are
- the 'Task' queue (sending tasks to the backend to be executed by the 'aegir' user from the command line, such as installing, deleting, enabling, migrating sites and platforms)
- the 'Cron' queue, which, when the Cron feature is enabled, runs cron on your site in batches.
The task queue that you'll see upon installation shows all the tasks that Aegir has recently run, or is about to run. It shows the last 5, but there is a link to see the full list of historical tasks.
After installation there will be as many as three tasks in the task queue:
- a Verify task for the main 'server' node,
- the Import task of the main Aegir site itself, and
- the Verify task of the main Aegir 'platform' that hosts the Aegir site.
These tasks are kicked off during the actual Aegir installation.
A task is colored
- a neutral blue-grey if currently queued but not running yet
- white with a spinning wheel if currently in the process of being run
- green if completed successfully
- red if there was an error
It's worth checking the queues regularly to see that the Task or Cron queues are being run regularly. If not you may have a problem with the cron setup on your server - see the FAQ.
Navigation
Underneath the Summary is the standard Navigation block in Drupal.
Content
The main content body in the Aegir interface lists
- your sites when on the frontpage or viewing the 'Sites' tab
- your platforms when viewing the 'Platforms' tab
- your servers when viewing the 'Servers' tab
The main content area also is where nodes are viewable or editable, such as viewing more information on a site, platform or server, or editing/creating new entities of these types.
Modalframe
When clicking on a task's 'Run' or 'View' buttons, a modalframe dialog is loaded in the browser. This is to provide a fluid, attractive experience for the user without requiring them to leave the current page or node to perform operations on a site. When the action is performed on the task, the modalframe will close and the user will be returned to the page they were on when they clicked that button.
Your site and Aegir
So, you've provisioned a site using Aegir, and you're poking about looking at the system and what it's done.
A Drupal site managed by Aegir is really mostly the same as any other Drupal site. However, there are a few very minor differences that might surprise or confuse some users.
None of these things adversely affect the running of your site - they are designed to actually make your life even easier.
Multisite
Aegir installs sites on a platform, which means it places the site directory in /var/aegir/platforms/(yourplatform)/sites/(yoursite.com). Many sites can all be installed alongside each other inside a platform's 'sites' directory. This is a standard, built-in Drupal feature known as 'multisite', and it is not unique to Aegir.
Settings.php and cloaked credentials
Within a multisite structure where multiple sites share the same 'Document Root' or codebase on the system, administrative users on sites that have PHP access, are capable of manipulating the server into exposing the database credentials of other sites in that codebase with reasonable ease. This is a pretty serious disclosure of sensitive information, although it is only possible if users have PHP execution privileges. Any multisite installation of Drupal (even without Aegir) has the potential for this to be a problem.
But security is very important to Aegir, and that's why special precautions are enabled by default to avoid this issue.
Essentially, when Aegir installs a site and creates the settings.php for that site, it 'masks' or 'cloaks' the database credentials, replacing them instead with environment variables fetched from the $_SERVER array in PHP.
The credentials are instead set in the site's Apache 'vhost' configuration file, which is stored outside the 'Document Root' or platform codebase, and is therefore safe from prying eyes by administrative users.
Apache/PHP is able to read the environment variables from the vhost and understand the settings.php's mask, allowing your site to work as normal.
If this feature confuses you or is inconvenient, you have a few options available to you:
- Run Nginx (which doesn't cloak the credentials)
- Set
$options['provision_db_cloaking'] = FALSE;
in the site's drushrc.php and then re-Verify the site - Stop worrying and enjoy the extra security Aegir provides you
Also note that when you run the Backup task against a site, it actually uncloaks the credentials of the settings.php that it saves in a tarball with the rest of the site. The benefit of this is that you are able to take the backup tarball and deploy it on a non-Aegir server, using standard Drupal, and your settings.php will be in 'normal' Drupal form for that environment. This also makes it easier to 'import' sites from previous-Aegir installations, into new Aegir installations if need be.
'Files' rewrite rule
Everyone knows about the inconvenience of having to place an img src
in your node that looks like this:
<img src="/sites/www.example.com/files/mycutecat.jpg">
If you ever deploy this site under a new URL or for whatever reason this path changes, those images or links will become broken on the site. That's because this information ends up stored in the database, and it can be a pain to fix up.
When you Migrate or Clone a site with Aegir, some paths in the database are automatically updated where they are consistent enough to be able to be scripted by Aegir (such as those in the files table).
However, Aegir can't fix up actual node body content, as that would be risky, and as a rule, Aegir doesn't like tampering that deeply into your data. Aegir exists to help you manage your data, but not manipulate it except where absolutely necessary.
Fortunately, what Aegir can offer is adding a '/files' redirect in the Apache vhost configurations. This pattern-matching rule allows you to enter this into your node instead:
<img src="/files/mycutecat.jpg">
and you can expect that to actually point to the same place. When you clone this site or rename it to a new URL, this path doesn't actually need to change, because the Rewrite rule will be modified to point to the new location, thus avoiding breakages.
It's not perfect, and people are often frustrated with Aegir not 'just fixing' everything like this, despite the fact the above is more than what Drupal does for you out of the box!
Fortunately, it's expected that Drupal 7 will improve this sort of thing altogether and things will hopefully become easier.
The .htaccess
Using a .htaccess with an Allow Override all
directive in Apache can be a major performance killer, because it requires Apache to stat each subdirectory of the codebase looking for overrides in .htaccess.
As a result, Aegir disables the reading of the Drupal .htaccess in the runtime environment.
This does not mean that the .htaccess is not needed. Instead, when you run the Verify task against a Platform, the .htaccess is studied by Aegir and its contents are copied into the platform-wide Apache vhost configuration, typically located in /var/aegir/config/server_master/apache/platform.d
Need to make a modification to the .htaccess? Simple: you can simply edit it in-place as you normally would, but you must re-Verify the platform in Aegir afterward, in order for those new or modified settings to be 'loaded in' to the platform vhost file.
The end result is improved performance for your sites, without losing any functionality, as you can still customise the .htaccess to your liking.
Permissions
Permissions in Aegir generally follow these rules:
- Everything is owned by aegir:aegir, and the 'other' bit is read-only in most cases
- Things that need to be written to (uploads) by the web server, have the 'group' bit of 'www-data' (or whatever the relevant user/group is on your system) with the group bit writable
- Anything that is sensitive information, such as the data in the configuration files, are owned by aegir:aegir or aegir:www-data when the web server needs to read those files, with no read access by the 'other' bit.
The use of the 'aegir' user on the system is vital for normal Aegir functionality, however it can be confusing to users who wish to edit files (say, theme development on a site, or downloading new modules) as a user other than Aegir.
The methodology recommended in these situations is:
- Add a standard user to your server, say, 'john'
- Add that 'john' user to the 'aegir' group:
adduser john aegir
- By default, Aegir sets the 'modules', 'themes' and 'libraries' directories so they are aegir:aegir with the group bit writable. This means that any user on the system who is a member of the 'aegir' group, is then able to add, delete, or modify files within those directories.
The drushrc.php
You may have noticed a file called 'drushrc.php' on your system, stored alongside the settings.php of a site (and another in the root-level directory of your platform or Drupal core codebase). What's this about?
The drushrc.php is a file that contains important metadata about your site or platform. The significant portion of the file contains metadata on all 'packages' (modules, themes, install profiles and libraries) in that part of the system, including their version numbers and whether they are actually enabled or not.
This file is generated by Aegir whenever a platform is verified, and whenever a site is installed or verified.
Paradoxically, settings from the existing drushrc.php are able to be 'read in' by Aegir when performing those tasks and many others.
The drushrc.php generally should not be edited unless you know what you're doing. As previously discussed above, some supported configurations such as the disabling of the 'cloaking' of database credentials in a site's settings.php, are able to be set in the drushrc.php to provoke the system into modifying how that site is to be treated.
@TODO: Drush aliases
Installing a new website
Creating a new website using Aegir is really, really easy, and is probably one of the most exciting aspects of Aegir. The very idea of clicking a button and suddenly being able to visit a new site may even be the prospect that drove you to installing Aegir in the first place.
To create a site you must have created a Platform first, which is the codebase, typically a copy of Drupal core, on which to place your new site. Read the Platform documentation if you have not yet completed this stage.
The Site form
Much like your Platform, a site is represented as a node in the Aegir frontend. To create a new site, you must create a new Site node.
The site node is a form requiring various attributes to be filled out in order to accurately advise the Aegir system about what kind of site you wish to create. These fields commonly are:
- Domain name - the site name. This becomes the title of the node.
- Client - who is the client or owner of this site. This field really only is relevant if you have enabled the Clients feature to manage segregations of sites between clients.
- Install Profile - the install profile to use to install this site. This has a dramatic effect on the end result of your site
- Platform - the platform to install the site on. The list of available platforms is based on what profile you choose above (in other words, which platforms support that install profile. The choice of platform also implies which web server hosts that platform, so this is why 'web server' is not a selectable option in the form.
- Language - what language install the site with. This is dependent on the profile or platform chosen.
- Database server - which database server to install the database on.
Below is an example of the site form.
Installing the site
Once you have filled out the site form per your requirements, click Save to submit the form. An 'Install' task is automatically spawned and added to the task queue for the dispatcher to execute.
The backend does the hard work of essentially completing a standard Drupal installation automatically. It creates a database and database user for the site, executes the Drupal installation procedure, populates the database, and generates a web server virtualhost configuration file for the site (Apache or NginX depending on your environment). In the event that the platform is located on a remote web server, the data is synced across from the main Aegir site to the remote server.
Finally, it restarts the HTTP service to load the new site's virtualhost into the HTTP service's runtime environment.
A traditional 'welcome' e-mail containing the one-time login link is sent to the e-mail address of the 'Client' who owns this site. As well as that, the login link is generated and displayed on the new site node view (see below).
As you can see above, the successful installation of the site has provided a series of tasks available to be performed against this site. This task block is similar in appearance to the Task Queue block we previously discussed, but rather than showing a list of tasks either run, queued or processing, it instead displays the list of potential tasks that may be run for this site only.
We will discuss these tasks and more in the following sections.
Site Aliases
Introduction
Site aliases are helpful if you move content to a new domain, change domain names or simply want to make sure that http://example.com and http://www.example.com take users to the same place. You can implement site aliases with or without redirection. The differences between these two methods and how Aegir handles them is described below.
Enable Alias Support in Aegir
You have to enable alias support in Aegir as it is off by default. Assuming the URL of your Aegir front end is aegir.example.com, browse to aegir.example.com/admin/hosting/features
Check the Site aliasing box
Click Save configuration
Global Settings for Aliasing
After you have enabled the site aliasing feature in Aegir you can navigate to aegir.example.com/admin/hosting/aliases (see the menu item that is added under Hosting in the admin menu bar). Here you can set default site alias settings for all of your Aegir-hosted sites.
Site-Specific Settings for Aliasing
You can override the global settings for aliasing by configuring site-specific alias settings either when you create the site or later by editing the site settings.
Redirection Option
For users, one of the main differences that occurs when redirection is turned on is that the URL that they enter in the browser address bar changes. This reflects what the server is doing, which is in effect to redirect the request to the same site under the main URL, as opposed to serving the same site from under an alias.
User navigates to my-old-domain.com with redirection off
User navigates to my-old-domain.com with redirection on
Aegir alias management under the hood
Aegir primarily manages aliases in the virtual hosts file for each site (e.g. /var/aegir/config/server_master/apache/vhost.d/www.example.com)
When Aliases are in use, the virtual hosts file will have a ServerAlias directive for each site alias:
ServerAlias example.com
ServerAlias www.my-old-domain.com
In addition to the above directives, when redirection is off, Aegir creates a symlink in the platform/sites/ folder for each alias that points to the folder associated with the primary domain
lrwxrwxrwx 1 aegir aegir 15 Oct 31 23:25 example.com -> www.example.com
drwxr-xr-x 7 aegir aegir 4096 Oct 31 23:02 www.example.com
lrwxrwxrwx 1 aegir aegir 15 Oct 31 23:25 www.my-old-domain.com -> www.example.com
When redirection is enabled, Rewrite directives are added to the virtualhost to send http requests to the primary domain that the aliases are associated with:
ServerAlias example.com
ServerAlias www.my-old-domain.com
RewriteEngine on
RewriteCond %{HTTP_HOST} !^www.example.com$ [NC]
RewriteRule ^/(.)$ http://www.example.com/$1 [L,R=301]
The symlinks are not required in this case.
Resetting site password
The 'Reset password' task is a simple one: it allows you to generate a new one-time login URL for a site.
The one-time login URL is bound to the account of the admin user (uid 1) of a site. It cannot be used to reset the password of another user on the site.
This can be handy in cases where the password for the admin user of a site has been forgotten. The 'Reset password' task is re-runnable and provides a convenient method of re-gaining access to a website from within Aegir.
In normal situations, viewing a site node shows a 'Go to' clickable link that takes the user to the site in question.
When the Reset password task has been run on a site, the 'Go to' link automatically gets changed to become a 'Login to' link that is the one-time login URL.
Once the one-time login URL has been clicked, this link returns to being a 'Go to' link.
Migrating/upgrading and renaming websites
One of the most powerful features of Aegir is the way it can help you to upgrade large numbers of websites safely.
For example, you may have dozens of sites hosted on a drupal 6.18 codebase. But suddenly the 6.19 release comes out with vital security fixes. Previously you'd have to go to each site, back up the files and database, upload the new codebase, run update.php, check everything worked and then onto the next site.
With Aegir you simply click a button in the frontend and it handles everything. This is called 'migrating' sites in Aegir terminology, because it might be used in more cases than simply upgrading the main codebase. For example you may have different 'platforms' with different mixes of contrib modules and themes and charge clients for tiers of 'basic', 'advanced' and 'gold' packages. When they choose to move from basic to advance, you simply migrate their site.
Other examples of using Migrate include:
- moving a standard Drupal site to a Pressflow platform for performance enhancements
- renaming a site (giving it a new URL entirely)
- the Clone feature (discussed in later chapters) is a subsidiary of Migrate, except it doesn't move but copy the site.
How to migrate a site
Enabling the feature
Aegir version 2.x
- The "Site migration" feature is enable by default. So you don't have to enable it.
Aegir version 1.x
- By default, Migrate is not enabled in Aegir after a fresh installation, as it is an optional feature. This means that the 'Migrate' button is not immediately present in the list of available tasks for a site.
- To enable the Migrate feature, simply go to
/admin/hosting/features
and check the Migrate feature. Submit the form and then return to the site node, and you'll see that the Migrate task is now available.
Migrating a site - the Migrate form
In the Aegir frontend, click on the website that you want to migrate. As per usual, in the 'Site view' you are presented with some summary information about the site, as well as a list of available tasks. Click the 'Run' button for the 'Migrate' task. A modalframe dialog will appear in your browser.
The Migrate form has several options. These are:
Domain name (this is already the domain of the existing site. If you are intending to simply rename the site (give it a new URL altogether), you can simply change this field and leave the target platform be the 'current platform'. This will make the site accessible under the new URL.
Platform selection. A radio selection of available platforms on the Aegir system. Certain platforms will be un-selectable. This is because they contain 'Errors' - that is, they contain versions of packages (modules, libraries or themes) that are of an older version than that of the original platform. You cannot upgrade a site to older versions of the same package.
Upgrades, warnings, errors?
What's this about upgrades, warnings and errors on each potential platform to migrate the site to?
Up until now you might be forgiven for thinking that Aegir is simply automating the install of sites and creating Apache vhosts and databases.
The truth is, Aegir actually is a much more advanced tool than that. It keeps a running registry in its database of your Platforms and Sites and what modules, themes, libraries and install profiles are available on platforms and installed on sites.
Not only does it keep track of what these packages are, but it also keeps track of what version of a package is installed (say, Views 6.x-2.11), and whether it is actually an enabled module or not.
The Migrate task, despite its name, is the tool for upgrading your site, because the Migrate code actually invokes the command drush updatedb
in the backend, automatically. drush updatedb
is the equivalent of running '/update.php' in your browser against your site.
In this way, when the time comes (and it always does) to Migrate your site, Aegir is able to intelligently analyse the available platforms on the system and make a sane judgement on where you can upgrade your site to. The logic is as follows:
- Current platform has module X, schema version 6002 and 6003
- Site module has module X, schema version 6003 installed
- Target platform A has module X, schema version 6002. Site cannot be upgraded to an older schema version. An error.
- Target platform B does not have module X at all. Theoretically this is still a possible upgrade, but module X will probably be disabled by Drupal's upgrade logic during the process. This is a warning.
- Target platform C has module X, schema version 6003. Site is upgradeable, there is no change.
- Target platform D has module X, schema version 6004. Site is upgradeable. There is a module update to be performed.
In the logic laid out above, examples 4, 5 ,6, or platforms B, C and D would be listed as valid target platforms. Target platform D would actually upgrade the site's module to a newer version, which may involve database updates.
To view the schema differences between current and target platforms in this way, click the 'Compare platforms' link against each target platform in the form.
If your intention is to upgrade your site between versions of any package (this can include core or contrib!), you need to make sure the relevant packages are present and up to date on the platform you want to migrate to. Aegir can only try and make this judgement call based on the information you feed into the system by adding and verifying platforms with the correct components.
If you've had to download extra packages to the target platform to make them possible candidates for an upgrade, you may need to visit that platform node in the Aegir frontend, and run the Verify task against the site (this is a re-runnable task). This updates Aegir's registry of information about all the packages on the platform.
Migrating
Select the relevant target platform and click 'Migrate' to submit the form. The modalframe dialog will close and Aegir will spawn a new Migrate task into the Task queue, and process it on the next cron run.
The process of Migration in the backend occurs automatically, like all Aegir tasks, and is as follows:
- The site is placed into maintenance or 'site offline mode' as a precaution. Safety first!
- A 'backup' task is implied silently. The site and its database are backed up into a tarball. Still safety first!
- Depending on the settings sent to the backend (are we migrating the site under its current name, or just renaming the site, or renaming the site and migrating as well), various metadata regarding the site is updated
- If we are moving to a new platform (upgrading, not just renaming), the backup tarball that was made earlier, is used in a silent 'provision-deploy' command, which essentially unpacks the tarball into the target platform and creates a new database and database user for the site, importing the database dump that was made in the backup
- The deploy task above, invokes
drush updatedb
, effectively upgrading the site by calling all hook_update() functions. - Various bits and pieces are fixed up, such as paths to files in the 'files' table of the database.
- Caches are cleared
- A verify task is run, settings.php and apache vhost are updated to reflect the new path (new platform) and new database credentials, site gets brought back out of maintenance mode.
- If all went well and nothing has fatally errored here, we remove the old copy of the site from the previous platform (it wasn't touched til now, other than backing it up)
Potential errors?
If something goes wrong during all this work, the metadata reflecting where the site is located on the system (which platform it is on) and other information is reverted. The task will become 'red' in the task queue, indicating an error.
Aegir will attempt to restore the site's vhost and settings.php to reflect the original platform and database credentials, and bring the site out of maintenance, in an effort to return the site to a state where it was prior to the Migrate.
The user may be requried to manually recover from this error depending on what has occurred. Nonetheless, remember that a backup was made early on, and this can be used to restore the site easily.
Migration Tips
In the context of sites, migration is the task of moving a site from one platform to another. Migration is the Aegir-preferred method for applying updates to modules and certainly to core. In other words, to upgrade core or modules, create a new platform with the upgrades, then migrate sites to the new platform.
Ideally, a migration requires nothing more than a mouse-click on the Migrate button on the Site node in your Aegir for the site you want to migrate. If you've prepared your destination platform properly, a migration will be this simple. Proper preparation, therefore, is vital.
To prepare a new platform to be a destination for migration, consider at least the following:
- The directory structure for your modules and themes must be the same on the source and destination platforms. For example, if the modules on the source platform are in
then the destination platform must have its modules also in this directory.
../platform-abc/sites/all/modules/contrib
- You may need to clear the Drupal cache on the source platform to avoid migration errors. Upgrades to Aegir in the future may handle this automatically, so be aware of the version of Aegir you're running and whether it includes upgrades to address this issue.
Renaming the Aegir hostmaster site itself
From http://community.aegirproject.org/node/363 :
WARNING: Be sure to have a good backup (which you can restore) before attempting this.
- manually edit the crontab for aegir to stop the periodic execution of the command 'hosting-dispatch' (the drush command 'hostmaster-pause' did not exist on my system)
- delete the hostmaster site (drush @hostmaster provision-delete)
- change the alias (drush provision-save @hostmaster --uri=hostmaster2.example.com --aliases=hostmaster.example.com)
- deploy the backup (drush @hostmaster provision-deploy /var/aegir/backup/hostmaster.example.com...tgz)
- checked that all changes appeared correct - YES
- Reneabled the hosting-dispatch task by editing the crontab again
- ran a verify task against the new hostmaster site. Noticed that the site is listed on the site page with the old domain name. The verify failed.
- Found that the verify task had created a new entry in the hostmaster-6.x-1.7/sites directory for the old domain name. I removed this created directory. I found and edited (using phpMyAdmin) the relevant entries in the node and node revision tables for the hostmaster site to update the domain name (change the title field). The hosting_site_alias table may also need to be updated if you have site aliases.
- The verify task also seemed to have regressed the @hostmaster alias file (~/.drush/hostmaster.alias.drushrc.php) as it had the URI and aliases details from the old site back in it. I edited this file to return these items to their new values.
- The verify task now completed successfully and all seems to be working correctly.
Cloning sites
Aegir provides an easy method of making entire copies of a site. This includes the actual site files, modules and so on, as well as a copy of the actual database.
This feature is called 'Clone' in Aegir, because it is a method of duplicating a site with a new URL or 'site name'.
The feature is very closely linked to the Migrate feature because it is almost the same, except that rather than move the site, it leaves the existing site in place and just copies it to a new name.
For this reason, enabling the Clone feature also enables the Migrate feature.
Enabling Clone
The Clone feature is disabled by default on a fresh Aegir installation. To enable it, visit /admin/hosting/features
in your Aegir frontend, check the 'Clone' box and submit the form.
Now when you visit a site node in your frontend, you'll see there's a 'Clone' button in your list of available tasks for the site.
Cloning a site
To clone a site, simply click the Clone button. A modalframe dialog will appear with a form. If you are familiar with the Migrate feature, you'll likely notice the similarities between the two forms.
The clone form has these options:
- Domain name - this is the URL for the new site, which must be unique.
- Platform - this is the target platform to clone the site to. It may be the current platform or a different platform that meets the requirements for hosting this site (has all the correct or newer versions of relevant modules)
- Database server - this option is only available if you have a remote database server configured, otherwise it is implied.
- Site aliases - If you have the Site Aliases feature enabled, you are also able to set new Site Aliases for this clone at this point. If the original site also had aliases, you will have to change or remove the aliases that load in this form before you can submit it.
As with the Migrate form, you can view the package comparison table between the current platform of the original site and that of a target platform, by clicking the 'Compare platforms' link. You can only clone a site to another platform if it meets the requirements for successfully hosting that site. In other words, the target platform has to contain the same or newer versions of modules. If the packages are missing on the target platform, those missing modules may be disabled.
Once you have made your selection and submitted the form, a 'Clone' task is spawned and added to the Task queue ready for dispatching.
What Clone does
The Clone task makes the following actions in the system:
- Makes a backup of the original site. This tarball will be used to 'deploy' a copy of the site
- Generates a new Drush alias for the new site
- Deploys the backup tarball to the new location as a new site
- The new site is imported and verified into the Aegir system, and relevant configurations are saved (HTTP vhost file, etc)
When might I want to use Clone?
Clone can be a very useful tool in a variety of situations. Commonly, people use Clone for:
- Testing what a 'Migrate' might do to a site, without migrating it. In other words, Clone can be used as a simulation tool to anticipate results of upgrading a site to a new release or build safely.
- If you have a 'template' site, often with a custom install profile, and anticipate having to generate multiple sites that are very similar, you can use Clone to rapidly do this.
- Cloning a live site to a 'development' version of a site, especially to a development platform residing on a remote 'development' web server, can be a useful and fast method of working on a site by ensuring the dev environment has the latest database and files from production.
Is there any relationship between the original and cloned site?
Currently there is no real relationship between the original site and any of its clones.
In future, Aegir development is likely to develop natural relationships between these sites, which will allow for 'rules' to be established, for example
- being able to regularly 're-clone' a site 'over the top' of previously-made clone, or
- automatically schedule clones after the original site has been migrated (upgraded) to keep a clone 'up to date' with the original.
Other uses may develop in time.
Disabling and Enabling sites
Aegir provides a method for sites to be temporarily 'disabled'. This means preventing all access to a site, but in a way that it can be re-enabled later.
Think of disabling a site in a similar fashion to placing a site under Maintenance mode, but that instead of a 'Site offline' message, a request to the site is redirected to a special page under the main Aegir frontend's URL, and a 'Site disabled' message is shown instead.
Aegir makes no presumptions about why you may wish to disable a site. Use this feature at your own discretion.
How to Disable a site
To disable a site, simply click on the site node in the Aegir frontend. In the list of available tasks for this site, click the 'Disable' button. A modalframe dialog will load, prompting you to confirm if you really wish to disable the site. Confirm and submit the form to spawn a 'Disable' task to the queue to take this site offline.
How to Enable a site
Once your site has been disabled, the list of available tasks has now radically changed in the site node: you now have only the option to Delete the site or Enable it again.
To re-enable a disabled site, simply click Enable. The site will be brought back out of hibernation and become live again.
Disabling before deleting
In Aegir's default configuration, you must first disable a site before you may delete it. This is meant as a precaution - if considering deleting a site, it forces you to think about whether you really want to, before accidentally clicking Delete and irreversibly removing your site.
But don't worry - Aegir always silently makes a backup of the site before disabling (or deleting) a site.
Backup and Restore
The Aegir system provides a convenient method of backing up sites, and restoring sites from those backups.
Like most other Aegir functionality, the Backup feature comes in the form of a task, aptly named 'Backup'. It is one of the core features of Aegir and thus is always available for a site, out of the box.
Backing up a site in Aegir is twofold: the site folder (containing the settings.php, files dir and so on) is added to a tarball along with a database dump of the site and stored in /var/aegir/backups
on the master Aegir server.
In the event that the website or database is a remote server, those relevant components are synced down to the master Aegir server to be tarballed up. Only the master Aegir server stores backups of a site. (This is known as a 'spoke' model, as opposed to a 'mesh' or distributed model, and suits the Aegir system at this time).
The tarball generated for a site can be used for a number of things: restoring a site from that backup via the 'Restore' task, or from the command line using the provision-deploy
command for advanced users as a method of quickly importing a site onto a system.
Other cases whereby a backup task is generated and its tarball used are when a site is being Migrated (upgraded) or being Cloned as a new site, but that is outside the scope of this chapter.
Backing up
When viewing the site node of a site in the Aegir frontend, click the 'Backup' button in the list of available tasks for this site.
A modalframe dialog appears, prompting for confirmation, as well as providing an editable Description field which you may fill out. This field is optional, designed only to aid you in future if you seek a reason for why a backup was made.
Once the form is submitted, a Backup task is spawned and added to the Task queue for dispatching by the backend.
The full output of a Backup task is viewable in the task log. Extra security precautions are built into Aegir so that the 'mysqldump' command uses file descriptors to hide exposing the database user's password in the running process and output of the task.
Restoring from a backup
Restoring a site from a backup is simple and follows the same pattern as backing up. Simply click the 'Restore' button from the list of available tasks for a site. A modalframe dialog will load, and the form offers a list of radio items. These are the backups that have been made for a site, in the format of 'Date/time - Reason'.
Where no custom reason was given for a backup, the message 'Generated on request' is shown instead.
Choose a backup to restore from and submit the form. A 'Restore' task will be spawned and added to the Task queue for dispatching by the backend.
What happens during a Restore task
The restore process
- puts the site under 'maintenance mode' temporarily (safety first!)
- generates a new backup of the site before restoring to it (still safety first!). This new backup will be available in the list of backups if you ever re-run the Restore task, and will carry the description 'Generated before being restored to a previous version'.
- uses the implied
provision-deploy
command in the backend to unpack the tarball, - create a new database and import the database dump into it,
- create a new database user,
- spawn a verify task silently, which will
- re-write the settings.php,
- sync the data if necessary to a remote platform,
- re-generate the web server's vhost configuration file for this site,
- finally reload the web server
Automatic backups and purge
There are contributed modules that can handle automatic creation and deletion (clean-up / purging) of backups:
For Aegir 2.x
For Aegir 1.x
Eventually these contributed modules will be incorporated into Aegir core.
It is also easy to set up a cronjob to automatically run backup tasks (that can still be used to restore from) for all your sites. To do this, download and set up mig5's bulk backup script.
Deleting sites
There may come a time when you want to remove a website from your Aegir system and your server altogether. As you'd expect, Aegir handles this for you too. It ships with a 'Delete' task that is capable of removing the site from the Aegir system, as well as carrying out backend tasks to remove the site from the filesystem and web/db servers.
The Aegir frontend is a Drupal site in itself, and sites are represented in that system as nodes. New users that are familiar with Drupal often make the mistake of thinking that the process of deleting a site is like deleting a node in Drupal, and attempt to delete the site node (or are confused by the fact we hide the Delete button for this reason).
This is an incorrect method of deleting a site, and doesn't actually spawn a task to clear away the site from the server at all. Below are the steps to correctly remove a site.
Method 1 (default) - Disable the site first
Part of this confusion can stem from the fact that when viewing a site node in Aegir, the Delete button is not actually present in the list of available tasks by default:
This is because, in the default Aegir configuration, you must run the Disable task against a site before you may Delete it. This logic is in Aegir to help users avoid accidentally deleting a site either by accidentally processing the task (despite having to confirm it first) or to prompt the user to think a little bit more on the decision before irreversibly blowing away the data.
The decision stems from Aegir development policy of being uncomfortable making such risky actions on your data. While it is possible to delete a site, we'd like you to really be sure that's what you want to do.
Once you Disable a site (see the Disable section prior to this chapter), the list of available tasks is modified to only offer an Enable or Delete button. At this point you can run the Delete task and permanently remove your site.
Method 2 (optional) - Delete without Disabling
The first method above is the default Aegir configuration. However, some users might be too confused by the lack of the Delete button altogether, which may not be ideal for your situation.
For this reason, Aegir also provides a configurable setting allowing a site to be Deleted without requiring a Disable first. This option makes the 'Delete' button appear in the list of available tasks, even when a site is currently in an Enabled state.
To activate this setting, visit /admin/hosting/settings
on your site and uncheck the checkbox that is titled "Require site to be disabled before deletion".
Re-visit the site node and you'll see that the Delete button is now present and clickable, alongside the Disable button.
The deletion process
When you click Delete and confirm your intentions in the modaldialog that loads, a 'Delete' task is spawned and added to the Task queue to await dispatch by the backend system.
The process runs through a series of steps to remove this site from your server. These are as follows:
- Make a backup of the site and its database for the last time (safety first, remember!
- Drops the site database
- Revokes the GRANT to that database for the database user (effectively deleting the database user)
- Deletes the site folder and everything inside it, from the platform that the site was hosted on
- Removes the Drush alias file for the site
- Removes the site's web server vhost configuration file
- Reloads the web server
- Sets the site's status to 'Deleted' in the Aegir system. Note: it does not delete the site node from the Aegir system. This is by design: the site node is retained in the Aegir database for historical purposes (you could customise Aegir by adding the ability to run a report or a View of all deleted historical sites, for example)
- The deleted status of the site, removes it from the list of sites in the 'Sites' page (and thus front page) of the Aegir system.
- Historical tasks run on the site remain in the Task queue history.
Manually deleting the site
Sometimes something goes wrong during an Install or a Delete task and a site doesn't install or get deleted successfully. The Delete task cannot be run or re-run on this site because Aegir has no way of knowing just how much of it got deleted (whether the files are still there but not the database, for example). Re-running the Delete task is likely to fail because some of these steps listed above will not exit successfully on completion.
At the time of writing, no task in Aegir exists to 'force' a removal of a site without having to bootstrap it first, so some manual steps are required on your part to remove this site.
- Manually remove the site files on the server if they exist (i.e /var/aegir/platforms/drupal-6.16/sites/my-failed-site.com
- Drop the database and revoke db user privileges associated with this site if it got created before failing (check the vhost configuration file's 'SetEnv' parameters, or the settings.php if the credentials are uncloaked, for the site's database name and user)
- Manually remove the Apache vhost file for this site if it is still present (in /var/aegir/config/server_master/apache/vhost.d/)
- Manually remove the Drush alias for this site if it is still present (in /var/aegir/.drush/)
- Enter the Aegir database from mysql and set the status of the node in the
hosting_site
table to -2 (deleted). For example, if your site node was nid 83 (the 'Edit' tab of the site node will tell you this), runUPDATE hosting_site SET status = '-2' WHERE nid = 83;
- Still in the Aegir database, delete the node record in the
hosting_context
table. For example, if your site node was nid 83 (as above), runDELETE FROM hosting_context WHERE nid = 83;
- Alternatively, if you want to even remove the node altogether from the system (not recommended), go to /node/83/delete in your browser on the Aegir frontend and delete the node. This will remove the site node and all associated task nodes from the system, as well as remove the entry from the hosting_site table in the database.
- If you are using the SSL feature and had Encryption enabled for this site, you may also need to manually delete the SSL certificate folder located in
/var/aegir/config/ssl.d/[domain of site you are deleting]
and/or/var/aegir/config/server_master/ssl.d/[domain of site you are deleting]
. Additionally you may also need to delete the entry for the domain you are deleting in thehosting_ssl_cert
table of the Aegir database. - If you are using the DNS feature you may also need to remove the site's entry in the zone file by running the following command:
drush @server_master provision-zone rr-delete example.com sub.domain A
- If your site has ssl support enabled you also need to remove ssl certificate from database issuing DELETE FROM hosting_ssl_cert WHERE ssl_key = 'your.site.domain';
- Re-verify the Platform node where the Platform is the one where your site was hosted on. This will regenerate some metadata whereby the Platform believed it still had that site contained within it.
If you need to manually remove the site's platform see the instructions here: http://community.aegirproject.org/node/27
Open ID tips and tricks
Allowing multiple people to access User 1
This approach depends on the OpenID Admin module. Here are the drush commands to download and install it, and set an openid access:
drush dl openidadmin
drush en -y openidadmin
drush openid-add 1 https://your.openid.server/your-open-id
This can allow you to simplify development workflow and easily control who has access to User 1, without giving away that account's username and password.
The best recipes for disaster and how to avoid them
This is a short list of all those bad things you should avoid while still learning how to use Aegir system properly, compiled from many real-world issues, so you could enjoy reading about creative ways to destroy your sites or Aegir system instead of experience it.
How to get all kinds of failed migrate or clone tasks?
Very easy. Simply use sites/domain-name/modules space for your code. This will guarantee many half-broken migrations with duplicates of the same site existing in two platforms, with fatal errors because some required include file in some module no longer exists, or not yet exists, as all paths to sites/domain-name/modules change on the fly when either domain-name or platform used changes, so some entries in your site's system table will cause either failed and reverted tasks in Aegir or some nice WSOD, until you will repair registry with http://drupal.org/project/registry_rebuild.
No, really, don't use sites/domain-name/modules for anything and save yourself headaches and frustration.
To help you understand this better, let's quote mig5 - Aegir core developer: "Aegir doesn't track site-specific modules/themes in /sites /$yoursite /(modules|themes) because it's not possible to ever upgrade those modules (they are tarballed up and moved with the site on a Migration). For this reason it's unwise to store site specific modules in this directory as you'll never be able to upgrade them if you do your upgrades properly (via Migrations, to new target platforms). To have site-specific modules not in that directory, you should learn install profiles, and store the modules/themes in /profiles /$yourprofile /(modules|themes) for a site using that install profile. (...) I don't see us ever handling the upgrades of modules in /sites/$site, since that entire dir gets tarballed up and moved across. In fact we made a deliberate design decision to literally ignore those modules in the Aegir package system." We highly recommend to read all comments there. There is also an excellent handbook entry you should read to understand this better.
How to break the site so even the Restore task will not help?
To make that happen you need to forget about correct upgrade workflow and good habits, use Backup task in Aegir to create a pseudo-safe copy of your site, upload new contrib modules to your new platform and then use Migrate task to move/upgrade your site to the new platform. To make it even worse, you could then try to use Restore task if anything went wrong with the migration and this will either fail completely or it will revert your database, but it will leave the site in the new platform – because Restore task will never move the site back to the old platform, so all your previous backups are immediately useless, and you are locked in the new platform without any chance to rollback automatically.
OK, so what is the correct workflow for sites upgrades in Aegir? It highly depends on how you manage your code, but some general rules are always valid and we will list them below:
- Create or choose new platform.
- Upload all your contrib modules and themes to the new target platform.
- Re-verify the target platform in Aegir.
- Clone your live site with some working subdomain in the old platform.
- Re-verify old platform and also just cloned site.
- Migrate cloned site to the new platform.
- Check if the cloned site works without any issues.
- If the step 7 above works, you can safely migrate also the live site.
Note: if this is a Drupal core version upgrade, it is always better to split the upgrade in two parts: first migrate to the new core, but using the same contrib modules versions, and then migrate to the final target platform with newer contrib modules. This may help if you experience failed upgrades with mysterious errors like "Call to undefined function node_types_rebuild()" without any hint on which module causes the issue.
How to break the site and lock it in the broken state in a one step?
This sounds frightfully, but it is really easy to do. All you need is to skip the steps with creating and testing the site’s clone in the existing platform first and simply use the Clone task to clone the site directly to the new platform. While this appears to be a handy shortcut, it is in fact one of the best recipes for disaster. Why? Because Aegir will not be able to properly check for all possible issues related to the contrib code and db updates to alert you before running the Clone+Migrate task. Yes, this kind of Clone task is in fact Clone and Migrate in a one step. It is very handy, but only when you already tested it with both old and new platform before. To stay safe, just avoid cloning the site to the different platform directly. Instead, clone it first in the existing platform, then migrate to the new platform, and finally re-verify it.
Setting up a Platform
What is a platform anyway?
Platforms are a type of node in Aegir and often the source of confusion for new users. This is because the term or concept isn't really used explicitly outside of Aegir - Aegir is a system that suddenly makes Platforms 'make sense' to have.
The simplest definition of a Platform is a copy of Drupal core. That's really it. When you download a copy of Drupal from drupal.org, the result is what Aegir thinks of as a 'Platform'. No sites exist on it yet.
Before you can create a site using Aegir, you must first define the Platform. This tells Aegir where to store the site directory, settings.php etc on the system.
In short: first you create a Platform, and a site 'lives' on that platform, in exactly this fashion:
- drupal-6.19 (Platform)
- /sites/yoursite.com/settings.php (site)
This abstraction is somewhat unique to Aegir in that it opens up a world of new opportunities for you. By managing Platforms or copies of Drupal core, and understanding what sites are on what copy of core, Aegir is capable of moving sites between platforms (which is effectively upgrading a site) - read more about this in Migrating/upgrading and renaming sites.
What else could be considered a platform?
Anything that is more or less a copy of Drupal core is something Aegir considers a 'platform'. This thus includes any Drupal distribution, such as OpenAtrium, Pressflow, Acquia Drupal, OpenPublish, ManagingNews, and so on.
The key difference between such distributions and a standard 'vanilla' copy of Drupal core is that these distributions tend to have * a custom install profile in the /profiles/ directory * a set of contrib or custom developed modules, libraries or themes shipped with the core or profile
If you build a copy of Drupal core and place your custom install profile in /profiles, this could be considered a Platform.
If you place your custom install profile in an existing Platform and re-Verify the Platform, that profile will now be recognised as an option when creating a site on that Platform.
Various 'versions' of Drupal, such as Drupal 5.x, Drupal 6.x or Drupal 7.x, are all considered separate Platforms in Aegir.
Drupal 4.x is not supported in Aegir.
Is this paradigm clear? 'Sites' are managed inside 'Platforms'. 'Platforms' are managed inside 'Servers'. 'Servers' are managed by Aegir. In this sense, Aegir manages all the rest too.
Getting a Platform onto your server
A number of techniques exist to put a platform on a server.
Drush
For a standard Drupal platform, the easiest is to simply use drush:
php /var/aegir/drush/drush.php dl drupal-6.19
Wget
You can simply use the 'wget' command, for example:
wget http://launchpad.net/prosepoint/trunk/0.23/+download/prosepoint-0.23.tar... -zxvf prosepoint-0.23.tar.gz
Version control
You could use CVS, eg:
export CVSROOT=:pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal
cvs login
cvs co -d aegir/drupal-6.15 -r DRUPAL-6-19 drupal
Manually
You could download it on your PC and scp or FTP the files up onto your server.
Drush Make
Drush Make comes installed on your Aegir system by default. You can use Drush Make to 'build' a platform, which does the job of fetching core and any other contributed or custom packages that you specify in a makefile.
You can even specify the URL or path to a makefile in the form you are given when adding a Platform node in the Aegir frontend, and Aegir will execute the Drush Make command in the backend and build it for you!
Explaining how to use Drush Make is outside the scope of this document. Consult the README.txt of Drush Make to learn the makefile syntax, or use a ready-made makefile available from the web. Some distributions such as OpenAtrium provide an example makefile for you to build the distribution with.
Platforms and remote web servers
Aegir has a 'spoke' model when it comes to remote servers, whereby the 'master' Aegir server keeps a copy of all platforms and sites and syncs changes outbound to remote servers that are running those platforms or sites, on tasks such as Verify etc.
Because of this, all platform names and paths must be unique, even across remote servers. This means you cannot have 'Drupal 7.0' in /var/aegir/platforms/drupal-7.0 on both Server A and Server B, because it can only exist once in that location on the master Aegir server. Platforms can't share the same name 'Drupal 7.0' because Aegir uses the platform name to define a 'context' by which it can refer to that server, and there can be no conflicts.
The exception to the unique path rule is when using web clustering (a collection of web servers running a platform), but even then, the platform is attached to the 'cluster' server and so is still 'unique' in this sense, that path cannot be reused for another platform running on another server somewhere else.
So when adding platforms to your filesystem and to Aegir, make sure that the platform is unique in name and path, so that other servers cannot try and use this reserved name/path for other platforms.
More on the platform name space
Migrating a site to a remote server illustrates this name space issue further: as an example; given that an Aegir hostmaster has two directories representing two platforms (and remembering that aegir does not migrate platforms, only sites):
/var/aegir/platforms/plat-local
/var/aegir/platforms/plat-remote
and the remote server also has:
/var/aegir/platforms/plat-remote
then having aegir hostmaster migrate a site from platform plat-local to plat-remote will result in the site files being:
1. moved from .../plat-local to hostmaster's local .../plat-remote
and
2. rsync'd to .../plat-remote
on the remote server.
It is important to understand this if you are importing a working Drupal site into Aegir's control and Aegir hostmaster is on a different server from the working Drupal site.
Do I have to have multiple platforms on my Aegir?
No, you can simply have one platform or copy of Drupal core and provision all your sites to sit on the one platform. However, eventually you will want to upgrade your sites to a new copy of Drupal core, and rather than replace your core files in-place, it's recommended to build a new Platform with the newer copy of core, and use the Migrate task to move your sites (upgrade) onto the new code.
Adding the Platform node
Now that you've got your Platform on your server, or you know where and how you're going to do it (say, with Drush Make), it's time to tell Aegir about your new Platform.
To do so, add a new node of type 'Platform' in your Aegir frontend. Typically this is most easily done by visiting Create Content > Platform in the Admin Menu.
The Platform node form has several fields required for giving Aegir information about your platform. These are:
- Name (a descriptive name for your platform. You very likely want this to be something like "Drupal 6.19".)
- Publish Path (the path on the filesystem where the platform is, or will be when Drush Make builds it. This must be the absolute path, for instance '/var/aegir/drupal-6.19')
- Makefile (the path on the filesystem to a makefile that will be used to create the platform.)
Once you have completed entering this information, you can click the Save button. A 'Verify' task will be spawned and added to the Task queue (visible in the right sidebar). The backend will then parse this new platform and build up a registry of information about it, such as what version of Drupal it is, as well as what versions of install profiles, modules and themes are present on that platform.
Certain system configurations, such as Apache configurations similar to the .htaccess file that comes with Drupal, will be written to the filesystem, permissions checked and adjusted where necessary, and services restarted.
Now that you have a platform or codebase that Aegir is aware of, you can now proceed to install or import sites onto that platform!
Verifying servers, platforms and sites
One of the common tasks available to all three of the major entities in Aegir (Servers, Platforms and Sites) is the Verify task.
You can think of the Verify task as a sort of routine 'sanity check' of that specific entity, with a sort of checklist of expected behaviour with which it runs through to confirm that the entity is operating as normal.
The Verify task also can be used to regularly 'sync' your platform or site with the Aegir package registry, to keep it up to date on whether any modules have changed (been removed or enabled/disabled). This will aid running Migration and Clone tasks, since Aegir needs to have a precise knowledge of what modules, themes or profiles are on the system before it can accurately judge whether an upgrade is possible.
It is recommended to run the Verify task on your entities routinely to keep them in sync with the Aegir database, and to run a Verify of your site, current platform and target platform prior to attempting a Migration or Clone task.
What happens in a Verify task?
Servers
- When you add a new server node in the Aegir frontend, a Verify task is spawned for the first time.
- the configuration directories are created if they don't exist and set to the correct permissions, or set to the correct permissions if they already existed but were incorrectly set
- if the server is the master server, some additional configuration and backup directories are created
- generates or updates a Drush alias file for the server.
- some configuration directories are synced from the master server to remote servers
- Checks that new databases and database users can be created
- Checks that the HTTP web server can be restarted
Platforms
- When you add a new platform node in the Aegir frontend, a Verify task is spawned for the first time.
- If the platform directory didn't exist on the file system and a Drush makefile was provided, Drush Make will be called to build the platform in that location
- If the platform directory didn't exist and no Drush makefile was provided, the task will fail with the error 'Does not contain a valid Drupal installation'.
- Checks that the permissions are correct, at least of the 'sites' folder of the platform directory (so we can create new sites here)
- Searches for existing sites on the platform that are not yet in the system. These sites will be 'imported' into Aegir.
- Scans the platform and builds or updates the package registry in the Aegir database.
- Generates or updates a Drush alias for the platform, and a drushrc.php file in the platform's directory root.
- Reads in the platform's .htaccess file into a platform-wide web server vhost configuration file.
- Restart the web server
Sites
- When you add a new site in the Aegir system, an Install task is spawned, This is contrasted with the above entities which spawn only Verify tasks initially.
- Scans the site and builds or updates the package registry in the Aegir database.
- Creates important directories in the site folder, such as 'files'. Sets or corrects the appropriate permissions
- Manages/updates any site aliases (symlinks, redirects etc)
- Re-generates the settings.php
- Re-generates the site's web server vhost configuration file
- Re-generates the site's drushrc.php and a Drush alias
- Clears caches.
You can run or re-run the Verify task at any time on a server, platform or site, and it is encouraged to do so.
Certain tasks, such as 'deploy' when a site is being imported, automatically imply a verify of the site.
Each Verify task, like all tasks generated by Aegir, emit a 'task log' available for review after the task has completed (or failed).
If a Verify task fails, and the task log does not provide enough information to help you resolve the problem, trying running drush @example.org provision-verify --debug from the command line to find out what caused the problem. Remember to replace 'example.org' with the URL of the site you are trying to verify.
Locking and Deleting platforms
Platforms, like sites, have tasks that can be performed against them, however these tasks are smaller in number and generally more simple.
Some tasks that are available to platforms out of the box are Lock/Unlock and Delete.
Lock
The lock task is simply a conceptual protection placed over a platform so that new sites can't be installed on that platform.
It doesn't actually modify anything on the filesystem or on the platform itself, but merely changes the status of the platform in the Aegir database. Once a platform is considered 'locked', it no longer appears in the New Site form when adding a new site.
You might choose to Lock a platform if there is something wrong with that platform and you want to prevent new sites from being added to it by one of your clients or users, or you want to 'hide' the platform for some other reason (perhaps awaiting a release date before allowing new sites).
Unlock
To unlock a platform that has been locked, simply click the Unlock button and the status of the platform will be switched back to an Enabled state.
Delete
The Delete task is similar to that of sites. The delete button allows you to completely remove a platform from your filesystem irreversibly.
This can be handy if you engage in a build management methodology that involves making regular new platforms to upgrade your site to. It is easy to amass large numbers of platforms on the filesystem, so the Delete task was born to deal with that problem.
A platform can only be deleted if there are no sites currently provisioned on it. The task will fail before deleting anything if it detects that there are sites in the 'sites' directory of that platform.
You must migrate these sites off to a valid new platform before you can delete such a platform.
Unlike site deletions, there is no backup made of a platform (because it's not bootstrappable by Drush), so be careful with this task: there is no going back!
It is recommended to make regular filesystem backups outside of Aegir altogether, and to use the Lock task to temporarily 'disable' a platform from use if you are unsure whether you want to delete it, but don't want it shown on the Aegir frontend.
Manually deleting a platform
Similarily to deleting sites, sometimes something goes wrong during a Delete task and the platform doesn't get completely removed. Often this is caused by permissions problems on the platform (i.e something owned by a different user that the aegir user can't remove).
The Delete task sometimes cannot be re-run in this situation. In which case,:
- Manually remove the platform files on the server if they exist (i.e /var/aegir/drupal-6.19)
- Remove the Drush alias for this platform if it is still present (in /var/aegir/.drush/)
- Remove the platform configuration file from /var/aegir/config/server_master/apache/platform.d (and the same for the other server_ directory if the platform was hosted on a remote webserver)
- Go to e.g. /node/123456/delete in your browser on the Aegir frontend and delete the node. This will remove the platform node and all associated task nodes from the system, as well as remove the entry from the hosting_platform and hosting_context table in the database.
Using drush_make to optimize workflow
Introduction
Once you understand why Aegir is a great tool for developing and managing Drupal web sites you inevitably start looking for ways to optimize workflow. mig5's article on Drupal deployments and workflows provides great insight on managing platforms, profiles and sites using Aegir, version control and the drush_make project.
The challenge of managing platforms and profiles
Whether you are using Drupal to develop bespoke web sites or vertical web applications with hundreds of instances (sites), it is not uncommon to be managing a mix of platforms and profiles. These comprise community-contributed projects that, in many cases, are frequently updated. To protect the integrity of their platforms developers like to explicitly control which version of which project they use and when. It can be time consuming to keep everything up to date. Many contributed projects are available in versions compatible with multiple Drupal versions so this mix of platforms and profiles can be complex.
Using drush_make to manage common and disparate code bases
One of the really powerful features of drush_make is that it can operate recursively. You invoke the drush make
command on a local make file but if drush_make fetches a profile or project that includes another make file then this is parsed and additional packages are fetched.
A typical way that developers leverage this feature is when they want to have some projects available to the entire platform and some that are specific to one or more custom profiles that are part of the platform. They achieve this by including a different make file with each profile repository.
Using make files with profiles works well when you want to manage distinct code bases that are specific to a site running under that profile. Only sites using profile 'foo' can see modules, themes or libraries located in subdirectories of the foo profile's directory. However, using the includes
option you can also reference a make file directly using a local path or a URL. The advantage of this is that you can maintain a make file that references the projects you use on every profile on every platform (e.g. views, cck, path etc.).
This is illustrated schematically below:
Example make files
Here are some example make files that illustrate the concepts described above:
<namespace>_<platform1>.make
core = 6.x
api = 2
; drupal core
projects[drupal][version] = 6.20
; platform projects
projects[webform][vesion] = 3.2
projects[workflow][version] = 1.4
projects[job_scheduler][version] = 1.0-beta3
projects[feeds][version] = 1.0-beta10
; platform theme
projects[acme_theme][type] = "theme"
projects[acme_theme][download][type] = "git"
projects[acme_theme][download][url] = "git@git.example.com:drupal/themes/acme_theme.git"
projects[acme_theme][download][branch] = "master"
; profile 1
projects[acme_profile1][type] = "profile"
projects[acme_profile1][download][type] = "git"
projects[acme_profile1][download][url] = "git@git.example.com:drupal/profiles/acme_profile1.git"
projects[acme_profile1][download][branch] = "master"
; profile 2
projects[acme_profile2][type] = "profile"
projects[acme_profile2][download][type] = "git"
projects[acme_profile2][download][url] = "git@git.example.com:drupal/profiles/acme_profile2.git"
projects[acme_profile2][download][branch] = "master"
; base make file
includes[acme_base] = "http://example.com/acme_base.make"
; patches
projects[superfish][patch][] = "http://example.com/superfish-add-acme-style.patch"
<namespace>_<profile1>.make
core = 6.x
api = 2
; ecommerce
projects[ubercart][version] = 2.4
<namespace>_<profile2>.make
core = 6.x
api = 2
projects[openlayers][version] = 2.x-dev
projects[ldap_integration][version] = 1.0-beta2
<namespace>_base.make
core = 6.x
api = 2
; utilities
projects[ctools][version] = 1.8
projects[jquery_ui][version] = 1.4
projects[jquery_update][version] = 2.x-dev
projects[modalframe][version] = 1.7
projects[advanced_help][version] = 1.2
projects[transliteration][version] = 3.0
projects[token][version] = 1.15
; content management
projects[imageapi][version] = 1.9
projects[imagecache][version] = 2.x-dev
projects[cck][version] = 2.9
projects[filefield][version] = 3.9
projects[imagefield][version] = 3.9
projects[diff][version] = 2.1
projects[date][version] = 2.7
projects[calendar][version] = 2.4
projects[views][version] = 3.x-dev
projects[views_slideshow][version] = 2.3
projects[content_profile][version] = 1.0
projects[og][version] = 2.1
projects[node_clone][version] = 1.2
projects[node_export][version] = 2.21
projects[node_import][version] = 1.x-dev
projects[nodequeue][version] = 2.9
projects[noderelationships][version] = 1.6
projects[reverse_node_reference][version] = 1.0
projects[nodereference_views][version] = 1.3
projects[page_title][version] = 2.3
projects[pathauto][version] = 1.5
projects[realname][version] = 1.3
; ui
projects[wysiwyg][version] = 2.2
projects[imce][version] = 2.1
projects[imce_wysiwyg][version] = 1.1
; site administration
projects[admin_menu][version] = 1.6
projects[devel][version] = 1.x-dev
projects[devel_themer][version] = 1.x-dev
projects[globalredirect][version] = 1.x-dev
projects[google_analytics][version] = 2.3
projects[xmlsitemap][version] = 2.0-beta1
; site building
projects[rules][version] = 1.4
projects[features][version] = 1.0
projects[context][version] = 3.0
projects[strongarm][version] = 2.0
projects[purl][version] = 1.0-beta13
projects[spaces][version] = 3.0
; site theming
projects[skinr][version] = 1.6
projects[superfish][version] = 1.6
projects[adaptivetheme][version] = 3.0-rc2
; libraries
; CKEditor
libraries[ckeditor][download][type] = "get"
libraries[ckeditor][download][url] = "http://download.cksource.com/CKEditor/CKEditor/CKEditor%203.4/ckeditor_3.4.tar.gz"
libraries[ckeditor][directory_name] = "ckeditor"
libraries[ckeditor][destination] = "libraries"
; jquery ui library
libraries[jquery_ui][download][type] = "get"
libraries[jquery_ui][destination] = "modules/jquery_ui"
libraries[jquery_ui][download][url] = "http://jquery-ui.googlecode.com/files/jquery-ui-1.7.3.zip"
libraries[jquery_ui][directory_name] = "jquery.ui"
Client management and access control
@TODO update this, it's been copied directly from GDO without modifications yet.
In some situations you may wish to allow different people access to Aegir, but restrict which sites they can manage.
Use Case 1:
As a developer you may simply wish to segregate your sites in Aegir by different clients for your own internal management reasons.Use Case 2:
If you have many staff working on different projects, you may wish to issue them with different logins and restrict which sites they can access on the Aegir system.Use Case 3:
As a drupal hosting company you might allow clients to sign up via your promotional website (using the 'Signup Form' feature in Aegir!), and then have their own account created on Aegir, with their newly provisioned site. They can then login and manage their own upgrades (by migrating to new platforms).
This functionality is provided by the Client feature in Aegir. Here's a brief guide to using it...
1. Enable the Clients Feature
If you didn't enable this feature during installation, you can do so now by choosing the following from the menu bar at the very top of the page: Hosting > Features. On this page, click on 'Experimental' to expand these feature options. You can tick 'Clients' and click on the 'Save Configuration' button.
NOTE: Experimental features are designed to be for developer preview only, although we do have reports of them being successfully used in production environments - but do so at your own risk!
2. Configure the Client Feature Options
From the top admin menu select Hosting>Clients. On this page you can configure how you want this feature to operate, with the following options:
Automatically Create User Accounts for New Clients
If you tick this option, whenever you setup a new Client within Aegir, they will have a drupal user account created for them to be able to access their sites within the Aegir site. You will definitely want to leave this unticked for Use Case 1, perhaps also for Use Case 2, but may want to tick this for Use Case 3.
Send Welcome Mail to New Clients
If this setting is ticked, the client will be sent an email when you create their drupal user account allowing them access to the system. Below this you can configure the email that will be sent.
Create Clients
Creating a new Client on Aegir is as simple as selecting Create Content>Client. Then enter their email address (twice for confirmation), Name and Organization. Now click 'Save'.
Depending on the options you selected above they might also have a user account created for them at this stage by Aegir, and then be emailed with their login details.
Otherwise the Client record will just be stored in the database and you will be able to assign sites to this client in the future.
Adding New Users to a Client
Perhaps you didn't initially allow clients to access their own Client area within Aegir and just kept it for your own internal use, or perhaps you now want to add additional users to a Client on the system.
To do this visit the Client page and select the 'Edit' tab. You'll see a list of allowed users. You can add users to this list by typing their username in the autocomplete field, or you can remove users by tciking the box next to their username and saving the form.
If the user you wish to add doesn't yet have a drupal user account on the Aegir site, you can create one as normal by going to the following on the top admin menu: User Management>Users>Add User. Then you can return to the client page and add their username to give them access to that client's area.
Clients Accessing the System
If you have selected above that new Clients should also have an Aegir account setup for them, then they will be able to access the Aegir site.
Within Aegir they will ONLY be able to see details about their own sites.
Administrator Manual
Aegir is a powerful system that provides you with a lot of opportunities and methods for managing your sites.
These methods come in the form of 'tasks' which can be applied against the various entities that Aegir manages. Such entities include:
- Servers
- Platforms
- Sites
This section 'Using Aegir' outlines all the various tasks you'll likely be performing against these entities and how to get the most out of your Aegir system. In doing so, it also represents the recommended 'best practices' associated with achieving certain results (such as upgrading sites).
This section can be read in any order. Choose a specific activity or task from the table of contents to get started or work your way through all the various tasks at your leisure.
Installation Guide
This page is being moved to http://docs.aegirproject.org/en/latest/install/install/
Aegir requires some special permissions to your server in order to automate some configurations. For example, when a new site is installed, the web server will be automatically configured (vhost) and restarted. Therefore Aegir cannot be installed on a shared hosting environment. Consult the system requirements to ensure you meet the necessary requirements.
Automatic or manual install?
The automatic install process through Debian packages is the currently recommended install process on Debian and Ubuntu platforms. It should just be a matter of adding the right sources to your APT repositories and running apt-get install aegir
.
The manual install process should take about 15 minutes for trained system administrators, between 30 to 60 minutes for others. The section contains detailed instructions on how to install Aegir manually on supported systems. It is aimed at system administrators and porters that want to control every step of the install process.
To decide which method is the right one, you may also want to review the operating system support page.
We also provide a troubleshooting guide to resolve common installation problems.
Note that, while installing Aegir does change some system configurations, it does only minor changes and keeps all its system configurations in one place so that it is easy to uninstall. It will not remove existing configurations. However, make sure you have backups before installing.
Automatic install and maintenance with Puppet
There is now a Aegir Puppet module that can further automate installation and maintenance of Aegir servers. It defaults to using the Debian packages, and is thus best-suited to Debian and it's derivatives (Ubuntu, Mepis, &.), but also supports (experimental) installation via the manual install process. This currently also only works on Debian, as it follows the documented installation procedure; however, it should be pretty straight-forward to adapt it to other OSes.
Development installs with Vagrant and Aegir-up
If you're installing Aegir for experimental or development purposes, you might be interested in Aegir-up, a Virtualbox/Vagrant-based local virtual machine automation system. It provides a simple method of building a complete Aegir system locally using the Puppet modules, and is, in fact, the primary development platform for them. In addition, you can build a dev version of Aegir from the git repos (with .git directories intact) to make debugging and writing patches and extensions easier.
Related documentation
System Requirements
Note that the following System Requirements are the same for Aegir automatic or manual install
- A system capable of running Drupal
- The Aegir system is entirely Drupal based, and has the same base requirements that Drupal does (with the exception that it won't run on Windows). See more notes on Unix and LAMP/LEMP requirements below.
- Your own server
- The low level of access required to be able to configure and run this system is very far beyond what is commonly available to users with shared hosting. A VPS from any popular provider such as Linode, Rackspace, Slicehost, Amazon EC, etc. will do fine. You will need root access to the server and the server needs to be dedicated to Aegir.
- A Unix-based operating system
- Aegir must run on some flavour of UNIX, because the majority of functionality in this system occurs in the back-end, through command line scripting. There are also several features (such as symlinks), that are not available to users on Windows. There are no plans currently to add Windows support. See the operating system support page for more information.
- Web server
- You will need at least one dedicated web server, running Apache. We generally work with Apache 2 but we should be compatible with the 1.x series. Aegir also supports the Nginx web server, but requires at least version 0.7.66 or newer. Since Nginx doesn't provide php-cgi or php-fpm (recommended) modules, you will need to install and run php-fpm server separately. You can find useful examples and tips in the third party Barracuda installer available at the barracuda project page.
-
N.B.: This third party installer is not supported by the core Aegir developers, but you can find helpful community support at the boa group.
- PHP 5.2 and 5.3
- Aegir depends on Drush 4.x, which requires PHP 5.2 or higher. Aegir 2.x depends on a minimun of Drush 5.10 (though Drush 6 is recommended), which requires PHP 5.3 or higher. You also need to have the command-line version of PHP to run Drush properly, and the MySQL extensions for PHP.
-
Given that PHP 5.2 has been deprecated since July 2010, we suggest using PHP 5.3 if possible. Note that while Drupal 6.x and above support PHP 5.3, some contributed third-party modules may still have problems with this version. Most often these cause warnings that can be safely ignored. Aegir and Drush themselves have no known outstanding PHP 5.3 compatibility issues, although you could have a lot of warnings in Drupal 6 due to ereg deprecation, see this issue for details. If you need to host Drupal 5.x sites, note that Drupal 5.x is not compatible with PHP 5.3 and above, and most likely never will be. See http://drupal.org/node/360605 (amongst other issues) for details. As such, Aegir has dropped support for Drupal 5 in versions 2.0+.
- Database server
- You will require a database server, obviously. Aegir currently only supports MySQL and MariaDB. It is preferable to use a dedicated (not shared-hosting) server since Aegir will create database users and will require the use of a MySQL
root
user. - Mail transfer agent
- Aegir requires an MTA (Mail Transfer Agent) installed on your webserver in order to be able to install new sites to your new platform. If you don't have an MTA, the site installation will fail with message like "could not send email". Additional messages will show that site has been removed because of this problem. To remedy the situation simply install an MTA like sendmail, postfix, or exim and do the minimal configuration.
- Other utilities: sudo, rsync, git and unzip
- Aegir installs itself via a Drush Make makefile that downloads via git if you want the bleeding edge code, or via wget if you want the latest official release. If you want the latest development version, and don't have the git program you will need to install it on the server.
-
The jQueryUI library is used in the Aegir UI, unzip is required to extract it. Sudo is required to allow the aegir user the limited privilege to restart the webserver when required. Rsync is used to sync files to remote servers.
- No conflicting Control Panels
- Other popular control panels such as Plesk, cPanel etc, are designed to manage all aspects of Apache configuration and other areas that Aegir also is intended to be used for.
-
Running Aegir alongside such control panels is not supported and very likely may cause you problems or difficulties installing or running Aegir. Filing bug reports that are caused by interference by another control panel will likely be closed unless the problem can be fixed without causing problems for other Aegir users. Proceed at your own risk!
- System requirements of popular Drupal distributions
- Some Drupal distributions, such as OpenAtrium, are specialized products that may contain unique prerequisites for optimal performance. Such examples may include raising the php-cli program's memory_limit to something higher than 64M.
-
Please note that this is not a requirement of Aegir but of the distribution you are trying to install a site on. Thus the Aegir documentation may not officially 'require' such performance settings, but be aware that Aegir may report errors if the system was under-resourced to complete such a task.
Automatic install on Debian
These are the installation instructions that are recommended on Debian. Aegir dependencies (Apache, MySQL, PHP...) are also automatically installed. If you are managing the installation from a remote Windows computer, well-known open source tools for this task are for example PuTTY (a SSH client for command line), and WinSCP (a SFTP client with easy text file editing).
If you wish to install Debian packages over an existing manual install, it's possible. See the Debian upgrade procedures.
Debian packages are uploaded to http://debian.aegirproject.org/ shortly after a release. We eventually want to upload those packages to the official archives, but this will take some adaptation and time to sponsor the packages in.
- 1. Requirements for automatic install on Debian
- 2. Adding the project repositories
- 3. Adding the archive key to your keyring
- 4. Adding backports for or manually installing Drush
- 5. DNS configuration
- 6. Manual sudo configuration
- 7. Manual installation of MySQL (on Ubuntu 12.04 LTS)
- 8. Installing Aegir
- 9. Custom Drupal distributions and make files
- 10. Troubleshooting the install
1. Requirements for automatic install on Debian
Basic Linux system administration skills
Root access to your server
- An up-to-date system and applications
-
Run the following command lines to update your system and applications.
-
aptitude update
aptitude safe-upgrade
- A configured Fully Qualified Domain Name (FQDN)
-
Such as aegir.example.com
- The
hostname
returned by the commandshostname -f
anduname -a
must resolve to the IP address of your server. -
After setting up your FQDN you must restart your server with a
reboot
command so your changes take effect. -
Note that newly created domain name usually take 24 to 48 hours to fully start working. This period, called propagation, is the projected length of time it takes for root name servers and cache records across the entire web to be updated with your website's DNS information.
- Other System Requirements
-
Find http://community.aegirproject.org/content/installing/system-requirements
2. Adding the project repositories
Use this command to add the Aegir package "Software Source" repository to your system:
echo "deb http://debian.aegirproject.org stable main" | sudo tee -a /etc/apt/sources.list.d/aegir-stable.list
To install a customized Debian package, see the developer instructions for the debian package. Other distributions are available for courageous people that want to try development versions.
2.x note: to install the development version of Aegir, you can use the unstable
or stable
distribution above.
3. Adding the archive key to your keyring
This repository self-signs packages uploaded to it (and packages uploaded are verified against a whitelist of trusted uploaders) using OpenPGP (GnuPG, to be more precise).
Use these commands to download and add the repository's PGP key, then update the package list on your system:
wget -q http://debian.aegirproject.org/key.asc -O- | sudo apt-key add -
sudo apt-get update
4. Adding backports for or manually installing Drush
If you are running Debian wheezy or later, or Ubuntu Natty 11.04 or later, you don't need to do anything here. The Drush package you need is available from your distribution's repositories.
If not, you should also configure backports repositories for Drush. Version 4.4 of Drush is now in Debian unstable, wheezy, squeeze-backports and Ubuntu Natty.
1.x note: if you are running Debian Squeeze 6.0 or Debian Squeeze 7.0, add the following line to /etc/apt/sources.list :
deb http://backports.debian.org/debian-backports squeeze-backports main
You might also have to add a proper Pin-Priority before this works. Create a file called drush containing the following and drop it into /etc/apt/preferences.d:
Package: drush
Pin: release a=squeeze-backports
Pin-Priority: 1001
Alternatively, you could download and install the squeeze-backports package for Drush 4.5 directly from: http://packages.debian.org/squeeze-backports/all/drush/download. Then you could install it with:
dpkg -i drush_4.5-2~bpo60+1_all.deb
2.x note: if you are running Debian Squeeze 6.0, to get drush-5.8.x and above, download php-console-table manually and install it
wget "http://ftp.debian.org/debian/pool/main/p/php-console-table/php-console-table_1.1.4-1_all.deb"
dpkg -i php-console-table_1.1.4-1_all.deb
You do not need to edit /etc/apt/sources.list or create /etc/apt/preferences.d/drush
Then run:
sudo apt-get update && sudo apt-get install drush
If you are running Debian lenny 5.0 or Ubuntu Maverick 10.10 or Karmic Koala or earlier, we recommend downloading and installing the Drush package manually. The version of Drush in the Ubuntu Universe repository for these versions of Ubuntu is outdated. If you are using Ubuntu Lucid LTS 10.04, you can install the Drush package manually or instead use Brian Mercer's PPA (Personal Package Archive) using the following command:
sudo apt-get install python-software-properties
sudo add-apt-repository ppa:brianmercer/drush
Now run apt-get update again to refresh the apt database.
sudo apt-get update
N.B. Since Aegir 2 requires Drush 5, which in turn requires PHP 5.3+, Drupal 5 sites are not supported in Aegir 2.
5. DNS configuration
Aegir requires a properly configured "FQDN" (Fully Qualified Domain Name) be assigned to the machine. In practice, this means that the hostname returned by the hostname -f
and uname -n
shell commands should resolve to the IP address for this server, and vice versa, with the resolveip
command (included with the mysql-server package).
For Ubuntu, /etc/hosts should have entries that look like:
::1 host.example.com host ip6-localhost ip6-loopback
127.0.0.1 host.example.com host localhost
123.123.123.123 host.example.com host localhost
To set this up in a virtual machine (e.g. Virtualbox), here are the steps:
- Create a new VM
- Go to settings->network. Enable Adapter 2, and set to "host-only"
- Install Ubuntu. Set hostname as FQDN during install
- You may need to add the lines `auto eth1` and `iface eth1 inet dhcp` to /etc/network/interfaces
If you have a virtual machine already setup and want to change the FDQN:
- change /etc/hostname using: `sudo hostname NEW_NAME`
- change /etc/hosts using: `sudo nano /etc/hosts` and change name
- reboot and test `hostname -f`, `uname -n`, `resolveip NEW_NAME`, `resolveip IP`
- YMMV - Your Mileage May Vary
6. Manual sudo configuration
If you are running Debian squeeze or later, or Ubuntu Lucid 11.04 or later, you don't need to do anything here. The Aegir package configures sudoers automatically.
If not, you will need to manually modify your /etc/sudoers file to add the following line:
echo "aegir ALL=NOPASSWD: /usr/sbin/apache2ctl" | sudo tee -a /etc/sudoers
The line above assumes that you have created a user aegir as specified in the installation instructions.
7. Manual installation of MySQL (on Ubuntu 12.04 LTS)
Please note that Ubuntu 12.04 LTS installs, by default, an insecure MySQL installation that contains an anonymous user grant, allowing anyone to login without a password. This breaks Aegir functionality.
If you are running Ubuntu 12.04, you should install MySQL manually, and then ensure it is installed securely:
sudo apt-get install mysql-server
sudo mysql_secure_installation
When running 'sudo mysql_secure_installation', answer 'Y' to 'Remove anonymous users?'
By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.
Remove anonymous users? [Y/n] Y
... Success!
Now you can proceed with installing Aegir below.
8. Installing Aegir
To install Aegir version 2, frontend and backend, use the following command:
sudo apt-get install aegir2
2.x note: Aegir version 2 is now stable. Read more at http://community.aegirproject.org/2.1
1.x note: to install Aegir version 1, use sudo apt-get install aegir
instead.
If apt-get reports that the aegir packages have unmet dependencies, then make sure that you have installed Drush (as explained above). On a debian system, you can force the install of Drush from the squeeze-backports repository:
sudo apt-get -t squeeze-backports install drush
Then install aegir using the apt-get command above.
This will prompt you for the required information (MySQL password, Postfix configuration...) and go ahead with the install.
During the Postfix configuration, the following options appear: "No configuration, Internet site, Internet with smarthost, Satellite system, Local only". That first text screen only allows to use the tab key to select "OK", and then the enter key to display a second screen where you can select one of the choices. The default is "Internet site", useful in most cases to enable the server to send email messages, for example to the admin.
At the end of the installation, you will receive an email message or, if the user "aegir" has been assigned with a local email account during the installation, the file /var/mail/aegir will contain the message. It will include a one-time login to your new Aegir control panel, that is a URL to copy into your browser so that you can set the password for the "admin" user.
9. Custom Drupal distributions and make files
If you have your own Drupal make file, you can go ahead with the above process, but change the make file to the one you want:
echo debconf aegir/makefile string /var/aegir/makefiles/aegir/aegir-custom.make | debconf-set-selections
apt-get install aegir
This allows you to specify the makefile path for your custom distribution of Aegir. To maintain these customizations, you'll need to ensure you do the same when upgrading.
An example aegir-custom.make file could look like:
core = 6.x
api = 2
includes[aegir] = "/usr/share/drush/commands/provision/aegir.make"
projects[] = module_filter
Note that for this to work, you may need the patch from this issue, allowing drush_make to reference absolute system paths. If drush make --version
<= 2.2 you need this patch.
After installing Aegir, you can reinstall the front end (hostmaster), with following commands:
sudo rm -rf /var/aegir/hostmaster-*.*
sudo su -s /bin/sh aegir -c "drush -y hostmaster-install --aegir_db_pass=$DB_PASSWORD --makefile=$MAKEFILE $DOMAIN"
su -s /bin/sh aegir -c "some command"
runs some command
in the /bin/sh
shell as user aegir
. sudo
runs the su
command as root - prompting for your user's password instead su
asking for aegir's password.
10. Troubleshooting the install
To make the install smoother, the install command is run without much debugging information, which can make diagnostics pretty hard. For this, there's a special environment variable you can set that will trigger debugging output. Install aegir with this*:
env DPKG_DEBUG=developer apt-get install aegir
You can build your own Debian packages from our repositories using those instructions.
Note: Prior to 1.1-4. the command was env DEBUG=yes apt-get install aegir
Automatic Installation on Ubuntu 11.04+ (QuickStart)
Here are simplified instructions for installing Aegir on a server, quickly and easily.
The fastest way to install Aegir is on the latest Ubuntu Server LTS (Long Term Support). Currently 12.04.
Command Summary
Here is every command, ready to copy and paste.
sudo su
echo "deb http://debian.aegirproject.org stable main" | tee -a /etc/apt/sources.list.d/aegir-stable.list
wget -q http://debian.aegirproject.org/key.asc -O- | apt-key add -
apt-get update
apt-get install aegir
It is highly recommended to read through the Installation Instructions step by step to give yourself the feel for how this is done. In addition, if you are not already familiar in Linux system administration, please read up on the subject. Standard linux security protocols should be followed.
Installation Instructions
Enter all commands in code boxes
exactly as they are written.
ALL commands should be entered as root
user until these instructions are complete. After that, you should never have to use the root user.
Self-installation of Ubuntu has you create an additional user in the admin
group, which means they can sudo. If you are using a cloud hosting service, it likely just sent you root user credentials. Server security and user management is up to you. Don't give away access to root or the aegir user unless you know what you are doing. Follow standard web host best practices
For a summary of all needed commands, scroll to the bottom of the page.
Fire up a new server
- Use Ubuntu 11.04 Server Edition for the smoothest and most reliable installation experience.
- Make sure it has at least 1024MB, preferably 2048MB of memory. Depending on your host you will have to configure a hostname. It can be convenient to call this 'aegir'.
- If you want the smoothest installation, create a brand new server. There will be the least chance of conflicts if you start with a brand new Ubuntu 11.04 server. If you use Ubuntu Desktop, you should be able follow these instructions without any problems.
- Run all commands as root. If you are not root, enter
sudo su
Add the project repositories and archive key
Use this command to add the Aegir package "Software Source" repository to your system:
Use this command to add the archive key to your keyring:echo "deb http://debian.aegirproject.org stable main" | tee -a /etc/apt/sources.list.d/aegir-stable.list
Then, finally, update your apt repositories:wget -q http://debian.aegirproject.org/key.asc -O- | apt-key add -
apt-get update
Install configure Aegir and dependencies
Once you have added the repositories, you can now fire off the standard debian installation command. For Aegir 1.x, the command is:
For Aegir 2.x, the command is:apt-get install aegir
This fires off the installation script for AEgir, along all dependencies including Apache, MySQ and PHP. You will be asked a number of questions about your server.apt-get install aegir2
- You will be asked to create a MySQL root user password. Make this long and random and type it down somewhere safe, you won't need it very often, but you will need it later on in the installation process.
- In Postfix Configuration: Choose Internet Site, unless you have a reason otherwise. When asked for the System mail name, pick either the hostname (default) or the domain name you will be hosting this server on.
- In Configure aegir-hostmaster, you will be asked to choose a "URL of the hostmaster frontend". This should be either the hostname (default) or the domain name the server will be hosting, with "aegir" as a subdomain. For example, "aegir.example.com". You may want to change this to whatever you prefer, just don't forget it as it will be where you use the Aegir front-end.
- After entering your domain, Aegir Hostmaster installation will ask you for the MySQL root password you created earlier. You did write it down, didn't you?
NOTE: The current DEB package requires you to enter the MySQL root password twice. When this script is done, if everything went ok, after a lot of other interesting information, you should see this:Aegir is now installed. You can visit it at http://test/user/reset/1/1329504351/eda205d9a27abde400a27cf160dff69a
***... frontend bootstrap correctly, operation was a success!
Setting up aegir (1.6-1) ...
Setting up libhtml-template-perl (2.9-2) ...
Setting up mysql-server (5.1.54-1ubuntu4) ...
Setting up php5 (5.3.5-1ubuntu7.7) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
root@test:~#
At this point, everything is installed. Visit the link the script provided you to check out the frontend. Switch to the
aegir
user to check out the backend:su - aegir
Once you are the aegir user, check out the drush site aliases it gives you.
@hostmaster
is the alias for the new front-end you created. You will get new site aliases for every platform and site you create.drush site-aliases
Give yourself access to the server
As the "Administrators Manual" can tell you, you should only manage the Aegir server from the backend as the
aegir
user.However, by default, the
aegir
user cannot sudo (except to restart apache). The aegir user also does not have a password. Therefor, the only way to become aegir is tosudo su - aegir
from a user that can.So, to finish the server, you should give yourself a personal account that you can use to login to the server with a password in case all of your SSH keys get lost.
To add yourself as a user:
Then fill out the little wizard it gives you.adduser yourname
//@TODO: Add some helpful notes about SSH keys and remote aliases.
Install SSH on your server, generating your SSH keys and install them on your Ubuntu server. (Directions to generate your SSH keys are assuming your PC is Ubuntu as well, Windows users look below for SSH directions)
If using Windows, you must use Putty, PuttyGen, and PuttyPageant to generate and use your SSH keys.
- Tutorial #1: http://the.earth.li/~sgtatham/putty/0.53b/htmldoc/Chapter8.html
- Tutorial #2: http://www.howtoforge.com/ssh_key_based_logins_putty
You will want to also be a part of the aegir and www-data groups so you can write to some of their files:
addgroup yourname aegir
addgroup yourname www-dataStart Using!
Now that aegir is installed, head to the User Manual page to get your first platform and site up and running.
Credits
This document was originally based on http://community.aegirproject.org/installing/debian but has been trimmed down to list only the steps you need to use on an Ubuntu server to get Aegir up and running as quickly and easily as possible.
Manual Installation
This page describes to process you need to follow if Aegir doesn't have packages for your distribution. We currently provide Debian packages and others should be coming, if you help! This manual assumes you are fairly familiar with the UNIX commandline interface, but should be possible to follow through if you copy and paste faithfully all steps of the procedure.
A note on supported systems
These instructions provide example commands for a Debian-like distribution, but should be fairly easy to adapt to other environments. This document is meant as a canonical reference that should work on every supported platform. It can also be used for people porting Aegir to new platforms or installing on alien platform for which Aegir is not yet packaged.
It currently contains special recommendations for CentOS, RHEL 6, Arch Linux and Solaris. Users of those platforms are also strongly encouraged to review the common installation problems that occur on those platforms. Aegir is also known to be installable (and was developed partly on) Mac OS X, but that process is so obtuse that it has a separate page for the first part of the manual (up to Install Aegir components).
Installing Aegir may seem daunting at first (which is why we provide automated installs through packages), but once you get around it, it's fairly simple. It follows those steps:
- 1. Review System Requirements
- 2. Install system requirements
- 3. Configure system requirements
- 4. Install Aegir components
- 5. Install the Hosting Queue Daemon
- 6. Checkpoint / Finished!
Note that these instructions setup a complete Aegir system. If you want to only setup a new remote web/db server, it should be sufficient to install the system requirements (step 1), configure them (step 2) and follow the Remote server how-to.
1. Review System Requirements
Find http://community.aegirproject.org/content/installing/system-requirements
2. Install system requirements
To install the required components, run the following command as root:
apt-get install apache2 php5 php5-cli php5-gd php5-mysql php-pear postfix sudo rsync git-core unzip
Note: replace apache2
with nginx php5-fpm
to install nginx on Ubuntu Precise or newer. Since Debian Squeeze doesn't provide php5-fpm, you will need to follow http://www.dotdeb.org/instructions/ before you will be able to install php5-fpm.
2.1. CentOS-specific configuration
yum install httpd php php-mysql php-cli php-gd php-process sudo rsync git postfix
For versions of CentOS previous to 6.0, you will need to upgrade to PHP 5.3 using those instructions.
Also for Centos minimal you should install cron (for queue and drupal cron) and unzip (for jquery.ui)
yum install cronie unzip
2.2. RHEL 6 specific configuration
RHEL 6 Server needs an additional PHP package to enable POSIX support. To find the package php-process you must enable the RHEL Server Optional channel. Once enabled, download and install the php-process-5.3.2-6.el6_0.1.i686.rpm.
You will also need to install the php-xml package if you are planning to use Aegir to manage Drupal 7 sites.
2.3. Solaris specific configuration
Solaris has this way of dealing with third party software that is... far from ideal. You will need to find the best way to install the following packages: apache2, git, sudo, mysql, PHP 5.2 and wget. unzip and sendmail should be part of the base Solaris install. The other applications should be available on the companion CDs or on sunfreeware.com.
In particular, git can be compiled easily by exporting the following environment::
export CFLAGS="-I/usr/sfw/include -I/opt/sfw/include"
export LD_LIBRARY_PATH="/usr/sfw/lib:/opt/sfw/lib:$LD_LIBRARY_PATH"
Then the compile instructions bundled with git should just be followed
directly. I had trouble installing the binaries, as git expects ginstall to be
available in the $PATH
. I ended up adding the source directory in the $PATH
, which works fine for most uses.
2.4. Arch Linux specific configuration
To install the required components, run the following command as root:
pacman -S apache php php-apache php-gd mysql postfix sudo rsync unzip git
Although all of the necessary apache modules and php extensions are installed at this stage, further configuration is required to enable and tweak certain features. Critically, virtual hosts are not enabled. It is worth examining the Arch Linux wiki page on LAMP server set up and verifying that more than one named virtual host functions properly.
If setting up for standalone development, see this useful wiki page to configuring postfix for local mail only.
To ensure Apache and mysql start when the machine boots, enable the httpd and mysqld daemons by adding them to the /etc/rc.conf
file:
DAEMONS=(... mysqld httpd ...)
3. Configure system requirements
3.1. Create the Aegir user
The provision framework of Aegir requires that the scripts run as a non-root system account, to ensure that it can correctly set the file permissions on the hosted files.
Also to ensure that the file permissions of the hosted sites are always as safe as can be, and especially to make sure that the web server does not have the ability to modify the code of the site, the configured system account needs to be a member of the web server group, in order to be able to correctly set the file permissions.
While you can choose another username, most aegir documentation assumes the Aegir user is aegir
, its home directory is /var/aegir
and the webserver group is www-data
.
Shell commands as root:
adduser --system --group --home /var/aegir aegir
adduser aegir www-data #make aegir a user of group www-data
3.1.1. CentOS specific configuration
CentOS requires special commands to create the user, use those instead:
useradd --home-dir /var/aegir aegir gpasswd -a aegir apache chmod -R 755 /var/aegir
3.1.2. Solaris specific configuration
groupadd aegir useradd -g aegir -G webservd -d /var/aegir -s /bin/bash -c "Aegir sandbox" aegir chown aegir:aegir /var/aegir
3.1.3. Arch Linux specific configuration
useradd --system --groups http --home /var/aegir --create-home aegir chmod -R 755 /var/aegir
3.2. Webserver configuration
Aegir supports two popular web servers, Apache and Nginx.
3.2.1. Apache configuration
Aegir assumes a few Apache modules are available on the server, and generates its own configuration files. The way we enable this is by symlinking a single file which contains all the configuration necessary. In Debian-based systems, you should symlink this file inside /etc/apache2/conf.d
that will be parsed on startup or alternatively you can place include that file in your apache.conf/httpd.conf. We prefer the former. In other systems there are similar ways to accomplish this. Consult your OS's documentation if unsure.
If you are on a Debian-based system, you will also need to enable the mod_rewrite module manually.
Run the following shell commands as root. First, configure Apache to enable RewriteEngine:
a2enmod rewrite
Finally, create a symlink from an apache configuration file to a folder within the /var/aegir/:
ln -s /var/aegir/config/apache.conf /etc/apache2/conf.d/aegir.conf
3.2.1.1. Ubuntu 14.04+ specific Apache configuration
Ubuntu 14.04 departs from Debian and previous Ubuntu Apache setup in that it doesn't use /etc/apache2/conf.d
any more and better separates out sites-enabled
from conf-enabled
configurations. So:
ln -s /var/aegir/config/apache.conf /etc/apache2/conf-available/aegir.conf
a2enconf aegir
Do not reload/restart Apache if prompted to after running these commands, it will fail.
3.2.1.2. CentOS specific Apache configuration
On CentOS, mod_rewrite is enabled by default and you can create the following symlink:
ln -s /var/aegir/config/apache.conf /etc/httpd/conf.d/aegir.conf
3.2.1.3. Arch Linux specific Apache configuration
On Arch Linux, mod_rewrite is also enabled by default. Add the aegir apache configuration include file to the httpd.conf file:
echo "Include /var/aegir/config/apache.conf" >> /etc/httpd/conf/httpd.conf
3.2.1.4. Other systems' Apache configuration
In other systems that do not have a conf.d directory, this could also work:
echo "Include /var/aegir/config/apache.conf" >> /etc/apache2/httpd.conf
N.B.:
- A standard umask of 022 is assumed. This is the default on most systems.
- For more information, see the common installation errors.
- For all OSes, the installer script creates the configuration file referenced by the newly created symlink/or file referenced in the Apache config file.
3.2.2. Nginx configuration
(If you just succeeded in installing Apache, please skip this section.)
Aegir assumes standard Nginx configuration is available on the server, and generates its own configuration files. The way we enable this is by symlinking a single file which contains all the configuration necessary. In Debian-based systems, you should symlink this file inside /etc/nginx/conf.d
that will be parsed on startup.
Please make sure your nginx installation is up and running before continuing. On Ubuntu 12.04 Server, for instance, you must edit /etc/nginx/nginx.conf and uncomment the line "types_hash_max_size 2048;" in order for nginx to start successfully.
Shell command as root::
ln -s /var/aegir/config/nginx.conf /etc/nginx/conf.d/aegir.conf
Do not reload/restart Nginx after running these commands, it will fail.
The installer script creates the configuration file referenced by the newly created symlink.
3.3. PHP configuration
Some complex installation profiles or distributions require a PHP memory limit that is higher than the default. To avoid common errors when installing sites on some distributions, the PHP command line tool should be configured to use 192Mb of RAM.
Change the memory_limit directive in /etc/php5/cli/php.ini to read:
memory_limit = 192M ; Maximum amount of memory a script may consume (192MB)
Most modern Drupal sites require around 96M or even 128M of RAM for certain operations. This is far more than what is provided by the default PHP configuration.
Change the memory_limit directive in /etc/php5/apache2/php.ini to read:
memory_limit = 128M ; Maximum amount of memory a script may consume (128MB)
If your distributions require more memory than these limits, then use some common sense and update it as appropriate to suit your individual needs.
For Aegir 3, make sure you've installed the required PHP extensions, particularly the image library (php-gd).
3.3.1. RHEL 6 specific configuration
The default php.ini configuration beyond the above changes also requires that the timezone be set for your location. Otherwise, you get fun errors and warnings during the host-master install step.
- sudo vi /etc/php.ini
- enter your password
- /zone (this will bring you to the date specific timezone module area
Remove the semi colon in front of date.timezone and enter your specific timezone.
[Date]
; Defines the default timezone used by the date functions
; http://www.php.net/manual/en/datetime.configuration.php#ini.date.timezone
date.timezone = Your Time Zone Goes HereRestart apache to compile these changes.
sudo httpd -k graceful
3.3.2. Arch Linux specific configuration
Make the following changes to the php.ini file (/etc/php/php.ini
):
Add :/var/aegir/
to the open_basedir
directive:
open_basedir = /srv/http/:/home/:/tmp/:/usr/share/pear/:/var/aegir/
Add date.timezone
value as per PHP's runtime configuration instructions - this is an example:
date.timezone = Europe/London
Modify the memory_limit
directive:
memory_limit = 192M
Uncomment
extension=posix.so
extension=mysqli.so
3.4. Sudo configuration
Next, we need to give the aegir user permission to execute the Apache2 command to restart the web server without entering a password.
Create a file at /etc/sudoers.d/aegir
and add the following:
Defaults:aegir !requiretty aegir ALL=NOPASSWD: /usr/sbin/apache2ctl
After saving, change the permissions on the file:
chmod 0440 /etc/sudoers.d/aegir
Note - the path to your apache2ctl program may differ from this example. On some systems it may also be called 'apachectl' instead of apache2ctl. Adjust to suit your own requirements.
3.4.1. CentOS Linux specific sudo configuration
For CentOS apache2ctl is apachectl and you should use this instead, as root::
visudo
This command opens an editor to allow you to edit the /etc/sudoers file. Add the following to the end of the file (specific directions cannot be given since this depends on what editor you're using):
Defaults:aegir !requiretty
aegir ALL=NOPASSWD: /usr/sbin/apachectl
Note - the !requiretty
bit is to make aegir able to run sudo even though it's not attached to a terminal. By default CentOS enforces requiretty so this exception is necessary.
3.4.2. Nginx specific configuration
For those using Nginx, set the sudoers line as follows
aegir ALL=NOPASSWD: /etc/init.d/nginx
3.5. DNS configuration
Aegir requires a properly configured "FQDN" (Fully Qualified Domain Name) be assigned to the machine. In practice, this means that the hostname returned by the hostname
and uname -n
shell commands should resolve to the IP address for this server, and vice versa.
If you only intend to use Aegir on a single server, it is acceptable for the resolved IP address to be the '127.0.0.1' loopback address.
If you intend to manage multiple servers using Aegir, you will need to make sure that the IP address is the public IP of this server.
You can add multiple entries to your /etc/hosts file for testing purposes, for example:
127.0.0.1 aegir.example.com example.com test1.example.com test2.example.com test3.example.com
Then you can use test1.example.com to create your first site.
3.6. Database configuration
Aegir supports MySQL right now. It is best to install the MySQL server using your Linux distribution's package manager.
Shell commands as root::
apt-get install mysql-server
To make sure that the Aegir backend, and all the possible web servers can reach your database server, you need to configure mysql to listen on all the public IP addresses available to it.
Again, as root, edit the MySQL configuration file /etc/mysql/my.cnf
configuration line to comment out by placing a # at the beginning of the line as follow:
Before
bind-address = 127.0.0.1
After
# bind-address = 127.0.0.1
Without this line commented out, MySQL will listen only on localhost for database connection requests.
Now you need to restart mysql, to clear any caches.
Shell command as root:
/etc/init.d/mysql restart
The installer will prompt you for your MySQL root user password. The root user will be used to make administrative tasks such as creating new databases, and granting and revoking access to those databases for sites.
Even though MySQL is now listening on all IP's, it will not allow invalid users to connect to the databases, without the correct user accounts configured.
If you are concerned about MySQL being accessible in this way, you can also configure your firewall to only allow incoming connections from certain addresses. This is outside the scope of this document however.
Note that Aegir will ask you for your MySQL root password. If you do not want to use your regular root password for Aegir, you will need to create another root account for Aegir using a MySQL command like:
GRANT ALL PRIVILEGES ON *.* TO 'aegir_root'@'%' IDENTIFIED BY 'password' WITH GRANT OPTION;
Note: If you are running your Aegir databases on a remote DB server, you will want to create this aegir_root user. The install will often fail if you're trying to use the root user on a remote database. See this issue for details.
3.6.1. Ubuntu, RHEL, Arch linux specific configurations
NOTE: If you are running either Ubuntu 12.04 LTS, RHEL 6 or Arch Linux, you should still install MySQL in the same way as above. However, once done, you must now remove the anonymous, passwordless login that those platforms creates by default. To do this, run:
sudo mysql_secure_installation
Otherwise, Aegir will fail to install and work at all. See this FAQ entry.
3.6.2. RHEL 6 specific configuration
In Red Hat, you may need to move a default configuration file from /usr/share/mysql/
to /etc/my.cnf
to view or modify any of the settings mentioned above.
4. Install Aegir components
Next step is to install the Aegir software components themselves, that is: drush, provision and hostmaster.
4.1. Install drush
Before installing Aegir proper, you first need to install Drush. This can be done through your operating system's package manager (Drush is shipped with Debian and Ubuntu currently) or by following the Drush README.txt file which has all the information for installing and using drush.
Note for 1.x users: you should use Drush version 4. Aegir 1.x does not support Drush 5. Also note there is a bug in Drush 4.0 and 4.1 so you should use a version of Drush between 4.1 and 5.
pear channel-discover pear.drush.org pear install drush/drush-4.6.0
Note for 2.x users: you need to install a minimum of Drush version 5.10, though Drush 6.x is recommended. In this case, you may be able to install the regular release:
Note for 3.x users: you need to install a minimum of Drush version 6,some time you need apt-get install php-console-table before you become the Aegir user:
pear channel-discover pear.drush.org pear install drush/drush
This should install Drush system-wide, but if you follow the manual install, you may end up with Drush in a non-standard location (traditionally /var/aegir/drush/drush.php
), in which case you will need to add that directory to your path or use the following symlink:
ln -s /var/aegir/drush/drush /usr/local/bin/drush
4.1.1. Arch Linux specific configuration
It seems that Arch's PHP environment needs to be modified for Drush:
mkdir /var/aegir/.drush cp /etc/php/php.ini /var/aegir/.drush/
Edit /var/aegir/.drush/php.ini
to remove the values after open_basedir =
, as this will any open_basedir
values are likely to cause Drush to fail.
4.2. Stop! Now become the Aegir user!
The remaining of this manual assumes you are running as the Aegir user. Things will go very wrong if you do not change your shell credentials to become that user. You can do this by running the following command as root:
su -s /bin/bash - aegir
If this fails because /bin/bash
doesn't exist, try using /bin/sh
.
4.3. Install provision
Once Drush is installed you should be able to install the latest recommended Provision release using the following drush command:
Note for 1.x users:
drush dl --destination=/var/aegir/.drush provision-6.x
Note for 2.x users:
drush dl --destination=/var/aegir/.drush provision-6.x-2.0 drush cache-clear drush
Note for 3.x users: replace provison-6.x-2.0 to provison-7.x
4.4. Running hostmaster-install
Once you have downloaded drush and provision, you can just install provision in the commands directory of Drush (either ~aegir/.drush or /usr/share/drush/commands), if that's not already done. Once provision is properly installed, you can install all other aegir components using the hostmaster-install command:
drush hostmaster-install
You will be prompted for the required information if not provided on the commandline. See the inline help for the available options:
drush help hostmaster-install
For example, to install the frontend on Nginx, use:
drush hostmaster-install --http_service_type=nginx
Note for 2.x users: Drush 5 has a commandfile cache which you need to clear before installing Aegir:
drush cache-clear drush
It is imperative that you provide a valid FQDN to the installer. This is used for database GRANTs. Remote web servers depend on the FQDN being resolvable in order to connect back to your Aegir master server if it is used as your database server for managed sites.
Upon completion of the installation, the traditional Drupal 'Welcome' e-mail will be sent to the e-mail address specified by --client_email=(your e-mail)
or if not provided as a command line switch, the address prompted by the installer process. This e-mail address will also be used as the default e-mail address of the first user and client in Aegir, but can be changed later.
There are other commandline switches available, documented in drush help hostmaster-install
, as usual.
4.4.1. Arch Linux specific configuration
drush hostmaster-install --web_group=http
5. Install the Hosting Queue Daemon
For Aegir 2.x installs, using the Hosting Queued Daemon (hosting_queued) is highly recommended. For Aegir 1.x, check out http://drupal.org/project/hosting_queue_runner instead.
These instructions will install the daemon to run as a regular service in /etc/init.d/. Instructions will vary according to platforms, but the following should work in Debian, running as root.
Install the init script in place
cp <hostmaster_platform_root>/profiles/hostmaster/modules/hosting/queued/init.d.example /etc/init.d/hosting-queued
Setup symlinks and runlevels
update-rc.d hosting-queued defaults
Start the daemon
/etc/init.d/hosting-queued
6. Checkpoint / Finished!
At this point, you have checked out all the code and setup your basic Drupal system (Drupal core, hosting, hostmaster and eldir) that will be the Aegir frontend and the backend system (provision and drush). Your filesystem layout should look something like this:
/var/aegir/hostmaster-1.x/ /var/aegir/hostmaster-1.x/profiles/hostmaster/ /var/aegir/hostmaster-1.x/profiles/hostmaster/modules/admin_menu/ /var/aegir/hostmaster-1.x/profiles/hostmaster/modules/hosting/ /var/aegir/hostmaster-1.x/profiles/hostmaster/modules/install_profile_api/ /var/aegir/hostmaster-1.x/profiles/hostmaster/modules/jquery_ui/ /var/aegir/hostmaster-1.x/profiles/hostmaster/modules/modalframe/ /var/aegir/hostmaster-1.x/profiles/hostmaster/themes/eldir/ /var/aegir/hostmaster-1.x/sites/aegir.example.com/ /var/aegir/config/server_master/apache.conf /var/aegir/config/server_master/apache/conf.d/ /var/aegir/config/server_master/apache/vhost.d/ /var/aegir/config/server_master/apache/platform.d/ /var/aegir/backups/ /var/aegir/drush/drush.php /var/aegir/.drush/drush_make/ /var/aegir/.drush/provision/
Variations on this are acceptable (for example, the Drush Debian package works out of /usr/bin/drush
and that's fine), but you are better to stick with the defaults if you really want to get through this.
The installation will provide you with a one-time login URL to stdout or via an e-mail. Use this link to login to your new Aegir site for the first time.
For troubleshooting this process and resulting install, see the common installation problems page.
You may also want to read on with what you can do with Aegir now that it is installed.
Mac OS X installation instructions
Apache
For Apache based installation hints see Apache / mySQL / PHP / Aegir
Nginx
Nginx is more performant than Apache, if you are interested in setting Aegir up using nginx Brian Gilbert from Realityloop has created a script to install everything that you need on a clean Mac (not already running anything on port 80), see OSX Aegir Installer on github.
Apache / mySQL / PHP / Aegir
This is a helper file to the canonical manual install process. It is aimed at helping you install Aegir on Mac OS X. Since PHP and MySQL support on OS X is fairly limited and complicated, a separate documentation page was created for that part of the documentation. You should follow this page all the way through and then proceed with the regular install, step 4: becoming the aegir user.
1. Special software requirements
While Mac OS X comes with Apache & PHP (and even MySQL on the Server version), the version of PHP shipped with 10.6 Snow Leopard is 5.3.x and thus may not work with Aegir (as of the 0.4alpha-era) and various other software. If you're running 10.5 Leopard, it may work out of the box, but I haven't tested it.
There are several different ways to get Apache, PHP 5.2, and MySQL 5 onto a Mac OS X machine. I give detailed instructions for MacPorts below, but if that's a bit more than you're ready to bite off right now, feel free to use an alternative approach.
One such alternative is MAMP. There is a good but outdated HOWTO for installing Aegir on Mac OS X 10.6 (Snow Leopard) using MAMP located here: http://groups.drupal.org/node/30270
MAMP stands for Mac, Apache, MySQL, and PHP and is the Mac equivalent of "LAMP". It is a self-contained package of all of these programs with a nice graphical installer and control panel. You can find it here: http://www.mamp.info/
MAMP is pretty straightforward, but it's also not very flexible (IMHO). While certainly not without its own headaches, MacPorts is a decently powerful way to sanely manage a healthy stack of open source UNIX software on your Mac. Since this is what I use, I'm going to assume MacPorts is in use for the rest of this HINTS file. I have also only tested this on Mac OS X 10.6 Snow Leopard.
If you don't yet have MacPorts installed, go here to get it: http://www.macports.org/install.php
Once it's installed, quit and re-launch your Terminal before continuing. Otherwise MacPorts won't yet be in your PATH.
The first two commands below are optional but recommended.
sudo port selfupdate
sudo port upgrade outdated
sudo port install apache2 mysql5-server git-core unzip php52 php5-posix php5-gd php5-apc +mysql5
php5-apc is optional, but highly recommended as it will significantly increase PHP performance.
Watch the output of the last port command carefully, as there are usually some boring tasks for you to perform once the install is done. You'll be wishing you were running Ubuntu/Debian and apt-get by the time you're done.
2. Configure system requirements
Next we'll create the aegir user and add it to the _www group. This part is very different on Mac OS X than Linux or most other Unices. Must be a NeXTism. The command we will use he is "dscl", which is a short for Directory Service Command Line. In OSX 10.3 and earlier, that command is "nicl" (short for Net Info Command Line). It is also possible to create the user using the "Workgroup Manager" utility included with OS X Server. To obtain Workgroup Manager for the OS X Client, download the "Server Admin Tools" from Apple. For example, for Mac OS X 10.6, the admin tools can be found at:
http://support.apple.com/downloads/Server_Admin_Tools_10_6
sudo dscl . -create /Users/aegir NFSHomeDirectory /var/aegir
Now you need to find the next spare UID to assign the user.
Here's how you find out on your system:
sudo dsexport users.out /Local/Default dsRecTypeStandard:Users
Then open the file users.out in a text editor, search for the highest 5xx user ID and add 1 to it (in your brain, not in the file). So if you find 506 but no 507, use 507. When you're done, delete users.out to be safe.
sudo rm users.out
Now assign this UID to the aegir user, replacing "5xx" with the UID.
sudo dscl . -create /Users/aegir UniqueID 5xx
!! If you're running Mac OSX Lion, you also need to assign PrimaryGroupID to the aegir user. sudo dscl . -create /Users/aegir PrimaryGroupID XXX
Set a secure password for the aegir user, as it needs shell access.
sudo passwd aegir
Create the aegir home directory and set its permissions.
sudo mkdir /var/aegir
sudo chown aegir /var/aegir
sudo chgrp _www /var/aegir
Add the aegir user to the _www group. This is the group Apache runs as.
sudo dscl . -append /Groups/_www GroupMembership aegir
Give the aegir user the ability to restart Apache.
sudo mv /usr/sbin/apachectl /usr/sbin/apachectl-apple
sudo ln -s /opt/local/apache2/bin/apachectl /usr/sbin/apachectl
sudo visudo
Go to the last line of the file and add the following.
aegir ALL=NOPASSWD: /usr/sbin/apachectl
Save the file and exit your text editor.
Next configure Apache to include the Aegir config.
echo "Include /var/aegir/config/apache.conf" >> /opt/local/apache2/conf/httpd.conf
Configuring your MySQL database and user accounts is the same as in the INSTALL.txt file. But you probably want to add the path to its executables to your user's PATH and the aegir user's PATH.
echo 'export PATH=/opt/local/lib/mysql5/bin:$PATH' >> ~/.profile
su - aegir
Password: (the password you setup earlier)
echo 'export PATH=/opt/local/lib/mysql5/bin:$PATH' >> ~/.profile
exit
nginx / MariaDB / PHP / Aegir (MEMPÆ)
The instructions that used to be here are now outdated, instead use the OSXAegirInstaller created by Brian Gilbert of Realityloop.
Centos 6.x Aegir Install Guide
There are 2 methods of installing AEgir on CentOS both are the same but one is scripted and the other is manual and is documented below.
Scripted
The script can be found at https://github.com/marafa/aegir/tree/master/version2
NB. There is preliminary work to fix selinux at https://github.com/marafa/aegir/blob/master/aegir_selinux.sh. Feedback is quite welcome as well as git pulls.
Explanation
Connect to the server via ssh as root user.
ssh root@000.000.000.000
Install system requirements
yum install httpd php php-mysql php-cli php-gd php-process php-pear php-mbstring php-xml php-soap sudo rsync git postfix tree wget cronie unzip mysql-server mlocate nmap samba samba-client samba-common vim
Note: The following packages are not required but are very useful to include git wget mlocate nmap samba samba-client samba-common vim
SElinux
Make sure Security-Enhanced Linux is disabled as it creates install problems.
vim /etc/selinux/config Make sure SELINUX=disabled
If was SELINUX=enabled then we need to restart.
shutdown -r now
Note: I am not sure if it can be enabled at the end I have never tried.
Create the Aegir user
The provision framework of Aegir requires that the scripts run as a non-root system account, to ensure that it can correctly set the file permissions on the hosted files.
Also to ensure that the file permissions of the hosted sites are always as safe as can be, and especially to make sure that the web server does not have the ability to modify the code of the site, the configured system account needs to be a member of the web server group, in order to be able to correctly set the file permissions.
While you can choose another username, most aegir documentation assumes the Aegir user is aegir, its home directory is /var/aegir and the webserver group is www-data.
useradd --home-dir /var/aegir aegir
gpasswd -a aegir apache
chmod -R 755 /var/aegir
Apache configuration
Start Apache
service httpd start
Make apache start automatically after reboot.
chkconfig httpd on
We need to create a symbolic link between aegir and apache.
ln -s /var/aegir/config/apache.conf /etc/httpd/conf.d/aegir.conf
PHP configuration
vim /etc/php.ini
Increase the memory limit as complex installation profiles or distributions require a PHP memory limit that is higher than the default (128M)
memory_limit = 192M
Set Date Zone to your time zone see http://www.php.net/manual/en/datetime.configuration.php#ini.date.timezone
date.timezone = “”
Sudo configuration
Next, we need to give the aegir user permission to execute the Apache2 command to restart the web server without entering a password.
visudo
Add to end of file
Defaults:aegir !requiretty
aegir ALL=NOPASSWD: /usr/sbin/apachectl
DNS configuration
Aegir requires a properly configured "FQDN" (Fully Qualified Domain Name) be assigned to the machine. In practice, this means that the hostname returned by the hostname and uname -n shell commands should resolve to the IP address for this server, and vice versa.
If you only intend to use Aegir on a single server, it is acceptable for the resolved IP address to be the '127.0.0.1' loopback address.
If you intend to manage multiple servers using Aegir, you will need to make sure that the IP address is the public IP of this server.
You can add multiple entries to your /etc/hosts file for testing purposes, for example:#> >vim /etc/hosts Add your ip and hostname
000.000.000.000 hostname
Database configuration
Start mysql
service mysqld start
Make mysql start automatically after reboot.
chkconfig mysqld on
Configure Mysql
/usr/bin/mysql_secure_installation
Recommended:
Set root Password
Remove anonymous users? y
Disallow root login remotely? y
Remove test database and access to it? y
Reload privilege tables now? y
Install drush
pear channel-discover pear.drush.org
pear install drush/drush-4.5.0
Check if drush works If you get PHP Fatal error: Class 'Console_Table' not found then
pear install Console_Table
Stop! Now become the Aegir user!
The remaining of this manual assumes you are running as the Aegir user. Things will go very wrong if you do not change your shell credentials to become that user.
su -s /bin/bash - aegir
Install provision
drush dl --destination=/var/aegir/.drush provision-6.x
Clear the drush cache
drush cache-clear drush
Run hostmaster-install
drush hostmaster-install
Manual install of a web cluster aegir using nginx
These are some really rough notes on how to go about creating a 4 server aegir installation (aegir, mysql, web1, web2).
Adapted from reading through the BOA project and my own experimentation.
** Note -- regarding the wildcard SSL, your sites will need some configuration in your settings.php or local.settings.php to check for the X-Forwarded-Proto headers. I can't recall if the wildcard SSL config.
These notes also assume the last Ubuntu LTS -- 10.04/Lucid.
aegirmysql: sudo apt-get update sudo apt-get upgrade sudo apt-get install vim mysql-server _USER="aegir" _DOMAIN="aegir.domain.com" _AEGIR_HOST="aegir.server.hostname" _AEGIR_HOST_IP="123.456.789.01" _AEGIR_PASSWORD="password" #AEGIR_DB_USER=aegir_root #AEGIR_DB_PASS=`echo $RANDOM:\`date\`:$AEGIR_HOST | openssl md5` echo "[client] user=root password=password" >> .my.cnf mysql -uroot mysql<<EOFMYSQL GRANT ALL PRIVILEGES ON *.* TO '$_USER'@'$_DOMAIN' IDENTIFIED BY 'password' WITH GRANT OPTION; GRANT ALL PRIVILEGES ON *.* TO '$_USER'@'$_AEGIR_HOST' IDENTIFIED BY 'password' WITH GRANT OPTION; GRANT ALL PRIVILEGES ON *.* TO '$_USER'@'$_AEGIR_HOST_IP' IDENTIFIED BY 'password' WITH GRANT OPTION; GRANT ALL PRIVILEGES ON *.* TO '$_USER'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION; FLUSH PRIVILEGES; EOFMYSQL ======================== # https://launchpad.net/~brianmercer/+archive/nginx # https://launchpad.net/~nginx/+archive/php5 aegircontrol: sudo apt-get update sudo apt-get upgrade sudo mkdir -p /var/www/nginx-default #php5-suhosin CATHOSTDEBDEPS="git-core git-doc mysql-client-5.1 vim nginx-custom drush postfix php5-cli php5-mysql php5-fpm php5-gd rsync unzip bzr patch curl" sudo apt-get -V install $CATHOSTDEBDEPS #postfix config already sorted sudo adduser --system --group --home /var/aegir aegir sudo adduser aegir www-data sudo chsh -s /bin/bash aegir #patch drush, re: ereg() #as root: echo "aegir ALL=NOPASSWD: /etc/init.d/nginx" >> /etc/sudoers ln -s /var/aegir/config/nginx.conf /etc/nginx/conf.d/aegir.conf #disable directives in nginx.conf: #types_hash_max_size #tcp_nopush #error_log invoke-rc.d nginx restart #install SSL cert to: /etc/ssl/private/domain.com.cert.pem cd /etc/ssl/private/ ln -s domain.com.cert.pem nginx-wild-ssl.crt ln -s domain.com.cert.pem nginx-wild-ssl.key #install SSL config to: /var/aegir/config/server_master/nginx/pre.d/nginx_wild_ssl.conf #TODO: also install for /var/aegir/config/server_aegirweb{1,2}.host.name #as aegir: cd ~ mkdir .ssh ssh-keygen -t rsa ln -s /usr/share/drush /var/aegir/drush mkdir ~/.drush cd ~/.drush wget -c http://ftp.drupal.org/files/projects/provision-6.x-1.3.tar.gz tar -zxf provision-6.x-1.3.tar.gz #htaccess password bit mkdir ~/tmp cd ~/tmp git clone --branch develop git://github.com/computerminds/aegir_http_basic.git #must be develop branch to use crypt() and for nginx support cp -r aegir_http_basic/provision ~/.drush/provision/aegir_http_basic cp -r aegir_http_basic/hosting ~/hostmaster-6.x-1.3/profiles/hostmaster/modules/hosting/http_basic_auth #set directory permissions? -- patch aegir/http_basic module to do so? _DOMAIN="aegir.domain.com" _USER="aegir" #_AEGIR_HOST=`uname -n` _AEGIR_HOST="aegir.server.hostname" _AEGIR_HOME="$HOME" _AEGIR_DB_PASS="password" _AEGIR_DB_HOST="mysql.server.fqdn" _AEGIR_VERSION="1.3" #_AEGIR_ROOT="$_AEGIR_HOME/hostmaster-$_AEGIR_VERSION" _ADM_EMAIL="admin@domain.com"" _WEBG=www-data _USRG=users #going vanilla echo "drush hostmaster-install $_DOMAIN --aegir_host=$_AEGIR_HOST --aegir_db_user=$_USER --aegir_db_pass=$_AEGIR_DB_PASS --http_service_type=nginx --db_service_type=mysql --db_port=3306 --aegir_db_host=$_AEGIR_DB_HOST --client_email=$_ADM_EMAIL --script_user=$_USER --web_group=$_WEBG --profile=hostmaster -d -v" drush hostmaster-install $_DOMAIN --aegir_host=$_AEGIR_HOST --aegir_db_user=$_USER --aegir_db_pass=$_AEGIR_DB_PASS --http_service_type=nginx --db_service_type=mysql --db_port=3306 --aegir_db_host=$_AEGIR_DB_HOST --client_email=$_ADM_EMAIL --script_user=$_USER --web_group=$_WEBG --profile=hostmaster -d -v cd hostmaster-6.x-1.3 echo "alive" >> healthcheck #enable aegir modules drush @hostmaster en hosting_web_cluster drush @hostmaster en hosting_alias drush @hostmaster en hosting_http_basic_auth #*** enable hosting client in features -- disabling client module cause WSOD on site add page # setup aegirweb{1,2} # test ssh to aegirweb{1,2} # add to known_hosts # NOTE: Aegir web clusters need to share the files, and private directories between web servers (also cache directory, if using boost module) # Setup provision hook for NFS links # http://drupal.org/node/1283738 mkdir -p /var/lib/sitedata/aegir chown -R aegir:www-data /var/lib/sitedata/aegir mkdir -p /var/lib/sitedata/aegir/cache chown -R aegir:www-data /var/lib/sitedata/aegir/cache # add web servers # add web cluster #TODO: Add DR web servers to cluster #TODO: Add WR, re: DR web servers & firewall # set date/time settings in Aegir #TODO: Logrotate webserver logs #TODO: Add an alias for the aegir user: #aegir: "admin@domain.com"" ======================== aegirweb{1,2}: #TODO: Check puppeted stuff, fix, etc sudo apt-get update sudo apt-get upgrade sudo mkdir -p /var/www/nginx-default CATWEBDEBDEPS="mysql-client-5.1 vim nginx-custom drush postfix php5-cli php5-mysql php5-fpm php5-gd rsync unzip patch" sudo apt-get -V install $CATWEBDEBDEPS sudo adduser --system --group --home /var/aegir aegir sudo adduser aegir www-data sudo chsh -s /bin/bash aegir #install SSL cert to: /etc/ssl/private/domain.com.cert.pem cd /etc/ssl/private/ ln -s domain.com.cert.pem nginx-wild-ssl.crt ln -s domain.com.cert.pem nginx-wild-ssl.key #install SSL config to: /var/aegir/config/server_master/nginx/pre.d/nginx_wild_ssl.conf #TODO: also install for /var/aegir/config/server_aegirweb{1,2}.host.name #as root: echo "aegir ALL=NOPASSWD: /etc/init.d/nginx" >> /etc/sudoers #as aegir: mkdir /var/aegir/.ssh cat aegir.aegircontrol.id_rsa.pub >> /var/aegir/.ssh/authorized_keys2 #TODO: Logrotate webserver logs ==========================
nginx / MariaDB / PHP-FPM Single Server Installation
Note: This installation process assumes that you're using a fresh install of Ubuntu 14.04 x64. If you use a lower version of Ubuntu, you may have trouble with this guide.
On most VPS providers, you'll be logged in as root initially. The installation process below assumes that you are logged in as root. Obviously, this is not a secure long-term solution, so once you're done with this guide, I suggest setting up public key authentication, turning off root login over SSH, and creating yourself a new unprivileged user. That's out of scope for this doc page, so you're probably on your own for that.
Finally, this document assumes that you're going to be installing aegir at aegir.example.com. Any time you see example.com, replace it with your domain.
1. Housekeeping
Make sure you're up to date:
apt-get update
apt-get upgrade
And that you have the the python-software-properties package (we'll need it later):
apt-get install python-software-properties
2. Install MariaDB
From mariadb.org:
MariaDB is a database server that offers drop-in replacement functionality for MySQL.
MariaDB is built by some of the original authors of MySQL, with assistance from the
broader community of Free and open source software developers. In addition to the core
functionality of MySQL, MariaDB offers a rich set of feature enhancements including
alternate storage engines, server optimizations, and patches.
Install MariaDB:
apt-get install mariadb-server
You'll need to set your root password for the MariaDB server
3. Install Nginx
Next, install Nginx and PHP-FPM:
apt-get install nginx php5-cli php5-mysql php5-fpm php5-gd
Create the default docroot for Nginx as well:
mkdir -p /var/www/nginx-default
4. Install all the other stuff
apt-get install git-core git-doc vim drush postfix rsync unzip bzr patch curl
When prompted for Postfix configuration, select "Internet Site", then use "example.com" for the System mail name.
5. Create the Aegir user
Easy:
adduser --system --group --home /var/aegir aegir
adduser aegir www-data
chsh -s /bin/bash aegir
6. Misc Configuration
Make sure the Aegir user is allowed to restart Nginx:
echo "aegir ALL=NOPASSWD: /etc/init.d/nginx" >> /etc/sudoers
Symlink Aegir's nginx configuration into place:
ln -s /var/aegir/config/nginx.conf /etc/nginx/conf.d/aegir.conf
Disable duplicated directives in /etc/nginx/nginx.conf
(the Aegir config specifies these values as well - if you do not disable them in the main nginx.conf, nginx will fail to restart). You can just remove (or comment them out with a "#") the lines that start with the following
types_hash_max_size
tcp_nopush
error_log
Then, restart Nginx:
service nginx restart
7. Install Aegir
IMPORTANT Switch to the Aegir user now: IMPORTANT
su - aegir
cd ~/
Download the latest Provision release:
mkdir ~/.drush
cd ~/.drush
wget -c http://ftp.drupal.org/files/projects/provision-6.x-2.1.tar.gz
tar -zxf provision-6.x-2.1.tar.gz
rm provision-6.x-2.1.tar.gz
Start the Aegir install process:
cd ~/
drush hostmaster-install aegir.example.com \
--aegir_host="aegir.example.com" \
--http_service_type="nginx" \
--aegir_db_user="root" \
--aegir_db_pass="[YOUR ROOT DATABASE PASSWORD]" \
--db_service_type="mysql" \
--db_port=3306 \
--aegir_db_host="localhost" \
--client_email="[YOUR EMAIL ADDRESS]" \
--script_user="aegir" \
--web_group="www-data" \
--profile=hostmaster
8. Optional Improvements
drupal.org/project/hosting_queue_runner
drupal.org/project/provision_boost
Common installation problems
There are a few things that can go wrong in the install.
- 1. Verify and install run everything through SSH
- 2. NameVirtualHost *:80 has no VirtualHosts
- 3. Making sure it works
- 4. Access by the server's physical IP address
- 5. CentOS firewall settings
- 6. CentOS cron requires restart?!
- 7. CentOS aegir.conf permission denied
- 8. Solaris cron issues
- 9. Drush execution path issues
- 10. APC issues
1. Verify and install run everything through SSH
Since Aegir has multi-server support, it is possible that you have a misconfigured "FQDN" and that aegir then tries to connect to the local server as a remote server. To check if you have a misconfigured server, run the following commands:
resolveip `uname -n`
If the command returns your IP address, you are all set. If it returns an error you will need to edit your /etc/hosts
file.
First line of this file looks like:
127.0.0.1 localhost
Simply add all domains you want to this line. e.g:
127.0.0.1 localhost aegir.example.com example.com
2. NameVirtualHost *:80 has no VirtualHosts
It does not hurt anything, but it can be annoying in your logs. This may disappear as soon as you define your first virtual site using Aegir. If it does not, you most likely have a second NameVirtualHost statement in your configuration someplace other than in the Aegir configurations.
If you are on a Debian system, that is usually a configuration fragment in /etc/apache2/ports.conf or in a fragment symbolically linked in /etc/apache2/sites-enabled and it is safe to comment out any NameVirtualHost statements you find there as this really is a large part of the job you have asked Aegir to do for you.
Once those are commented out, the message should disappear.
3. Making sure it works
Your new Aegir server will be installed with a single site named the way you specified in your install script. That's great, but you may have more paths to that same server.
When you try to browse to your server for the first time from a non-localhost browser you may get an addressing issue. If you do, make sure that you actually have the server defined in DNS and that the DNS server was reloaded. If it was reloaded and you use slave servers, make sure that the serial number in the zone file was incremented so that the slaves automatically reload.
If you already have multiple URLs in your DNS which resolve to the same Aegir, you should check them. For instance: if your DNS resolves both aegir.example.com and sitecontrol.example.com to your new Aegir server's IP, you need to make sure that both are accommodated. If they will both get the same physical hostmaster site, one should be set up as an alias to the other. If they are indeed going to be separate sites you will have to create a new site node with the other name as a virtual site.
4. Access by the server's physical IP address
Another "gotcha" that you may run into is http access using the actual IP itself. Remember that the IP is not picked up by the site vhost and will not match a ServerName or ServerAlias - so it gets picked up by the default vhost. This is just standard apache stuff, nothing Aegir or Drupal specific here, but quite the annoyance and a potential problem.
You can easily work around it by simply adding the IP as a Site Alias in the site node in either the Aegir site (or another site that you may have defined which you would prefer the IP to address).
You can tell if you have the IP problem by simply pointing your browser to the server by IP as http://999.999.999.999/ and seeing what comes up. If you get an install screen, you have the problem. You certainly do not want that install screen getting executed!
The good news is that fixing it is easy.
Simply log into Aegir, click on Sites, click on the site name that you want the IP to address, click on Edit and scroll down to Domain Aliases. Put the IP into the box, click on the Redirect checkbox so that Aegir instructs Apache to do a rewrite to the real domain name, and finally click Save.
If you do not have the Domain Aliases entry box, you need to turn that feature on. In that case, go to your administration menu at the top of the page, point to Hosting and then click on Features. Click the checkbox for Site Aliasing and then Save Configuration. The Aliases box will now appear when you edit a site. There is excellent information about the site alias feature in this handbook at http://community.aegirproject.org/node/60 with much more detail.
After the Verify job completes you should test your server again. Now when you address the server by it's IP address the URL should automatically change to the site you selected and the install screen never appears.
5. CentOS firewall settings
You may need to adjust CentOS's firewall settings to allow HTTP traffic on port 80. If you installed CentOS with a UI, enable "Firewall settings -- WWW (HTTP)".
Alternatively, another solution may be to edit /etc/sysconfig/iptables and add a rule accepting traffic on the relevant interface on port 80.
Afterwards, you can restart the firewall with this command:
Shell commands::
service iptables restart
6. CentOS cron requires restart?!
Also, in some configurations, it seems necessary to restart crond for the user crontab changes to take effect (very bizarre). For that, use:
Shell commands::
service crond restart
See http://drupal.org/node/632308 if you have more information about this issue.
7. CentOS aegir.conf permission denied
If you receive:
Starting httpd: httpd: Syntax error on line 209 of /etc/httpd/conf/httpd.conf: Could not open configuration file /etc/httpd/conf.d/aegir.conf: Permission denied
chcon -t httpd_config_t apache.conf
You can see this comment: http://drupal.org/node/1286926#comment-5028062 for a little more info.
Try the following if nothing works:
setenforce permissive
See this doc to make the change permanent: http://wiki.centos.org/HowTos/SELinux
8. Solaris cron issues
I had numerous problems setting up a proper cron job, as Solaris' crond seems pretty anal about what it accepts. The only way I could get it to work was to create a wrapper shell script that would be called using the simplest cron tab.
Crontab entry:
* * * * * /var/aegir/dispatch.sh
Content of dispatch.sh:
#!/usr/bin/bash
HOME=/var/aegir
LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/usr/lib/sparcv9:/opt/mysql/mysql/lib:/usr/sfw/lib:/usr/sfw/lib/gcc:/opt/sfw/lib
PATH=/usr/bin:/opt/mysql/mysql/bin:/usr/sfw/bin:/opt/sfw/bin:/opt/SUNWspro/bin:/usr/local/bin:/opt/csw/bin
export HOME
export LD_LIBRARY_PATH
export PATH
php '/var/aegir/drush/drush.php' --php=/usr/local/bin/php '@hostmaster' hosting-dispatch
9. Drush execution path issues
Solaris (and maybe others) suffers from the dreaded execution issues of drush:
Those can be worked around by hardcoding the --php executable on the
commandline path. Adding the proper shebang (#!/usr/local/bin/php
, for example) header and using a proper PATH
that includes the PHP executable also helps.
10. APC issues
If you are having trouble running APC with Aegir, try downgrading to APC 3.1.4. This can be achieved by the following:
sudo pecl uninstall apc
sudo pecl install apc-3.1.4
Ubuntu 10.04 Specific : How to downgrade to php 5.2 before install
As explained at http://community.aegirproject.org/installing/manual, the recommended version of PHP is 5.3, but some users may wish to use PHP 5.2 in order to host Drupal 5.x sites or to use modules which have PHP 5.3 compatibility issues. Should you choose to do so, the following documentation may be useful. What is explained here is specific to Ubuntu 10.04. It hasn't been tested on further versions, mostly because many sysadmins prefer to use the latest Long Term Support (LTS) version of Ubuntu, currently 10.04, on their production servers.
Follow this procedure :
sudo apt-get install python-software-properties
add-apt-repository ppa:txwikinger/php5.2
Create the file /etc/apt/preferences.d/php and input the following code in it :
Package: libapache2-mod-php5 Pin: version 5.2.10* Pin-Priority: 991 Package: libapache2-mod-php5filter Pin: version 5.2.10* Pin-Priority: 991 Package: php-pear Pin: version 5.2.10* Pin-Priority: 991 Package: php5 Pin: version 5.2.10* Pin-Priority: 991 Package: php5-cgi Pin: version 5.2.10* Pin-Priority: 991 Package: php5-cli Pin: version 5.2.10* Pin-Priority: 991 Package: php5-common Pin: version 5.2.10* Pin-Priority: 991 Package: php5-curl Pin: version 5.2.10* Pin-Priority: 991 Package: php5-dbg Pin: version 5.2.10* Pin-Priority: 991 Package: php5-dev Pin: version 5.2.10* Pin-Priority: 991 Package: php5-gd Pin: version 5.2.10* Pin-Priority: 991 Package: php5-gmp Pin: version 5.2.10* Pin-Priority: 991 Package: php5-ldap Pin: version 5.2.10* Pin-Priority: 991 Package: php5-mhash Pin: version 5.2.10* Pin-Priority: 991 Package: php5-mysql Pin: version 5.2.10* Pin-Priority: 991 Package: php5-odbc Pin: version 5.2.10* Pin-Priority: 991 Package: php5-pgsql Pin: version 5.2.10* Pin-Priority: 991 Package: php5-pspell Pin: version 5.2.10* Pin-Priority: 991 Package: php5-recode Pin: version 5.2.10* Pin-Priority: 991 Package: php5-snmp Pin: version 5.2.10* Pin-Priority: 991 Package: php5-sqlite Pin: version 5.2.10* Pin-Priority: 991 Package: php5-sybase Pin: version 5.2.10* Pin-Priority: 991 Package: php5-tidy Pin: version 5.2.10* Pin-Priority: 991 Package: php5-xmlrpc Pin: version 5.2.10* Pin-Priority: 991 Package: php5-xsl Pin: version 5.2.10* Pin-Priority: 991 Package: php5-mcrypt Pin: version 5.2.6* Pin-Priority: 991 Package: php5-imap Pin: version 5.2.6* Pin-Priority: 991
Then proceed Aegir install as explained at http://community.aegirproject.org/installing/debian
Post-install configuration
Alright, so you have installed Aegir, what else is there?
You have a lot of things you can do now:
- Get to know the interface
- Use drush make
- Work as the Aegir user
- Debugging Aegir
- Setup the hosting queue runner for lower waiting times
A lot more is available in the user manual.
Importing sites
It is possible to import existing sites into the Aegir hosting system, so that they can be managed by Aegir.
These instructions assume you've set up a new server with Aegir on it, and you want to import sites into Aegir from another server, or even from the same server, using backups.
There are basically three ways of importing sites in Aegir:
- from Aegir backups - simplest and fastest, scriptable
- from any single site, manually - most reliable, can import even non-aegir sites, hard to automate
- from a complete Drupal install - fairly reliable, can import non-aegir sites, could be automated but requires access to the database servers currently in use by the site
This section also regroups troubleshooting and other less orthodox techniques.
Importing from existing Aegir backups
If you need to import a site from an existing Aegir system, this can be done from an Aegir site backup.
Requirements:
- New Aegir version 0.4 alpha15 or later.
- Access to Aegir backups from the old server.
- A platform set up and verified that matches the platform on the old Aegir server.
1. Copy backups
First log into the new server via ssh - switch to the aegir user. Create a temporary directory somewhere, for example:
mkdir /var/aegir/tmp/migrate
Run a command to syncronize the new directory with the backups directory of the old Aegir hosting system. This command does the entire directory, but you may want to modify this if you have a lot of unneeded backups on the old system.
This command also assumes that you can shell into the other server as the aegir user. Modify as required.
rsync -aPv -e ssh aegir@old-server.com:/var/aegir/backups/* /var/aegir/tmp/migrate
Now you can re-run the above command just before you are going to migrate a site. This is handy if you have a lot of sites to migrate and each site requires attention. Simply make a new backup of the site in the old system, and rerun the rsync command.
2. Create the site context and deploy the backup
For brevity, I assume that drush can be executed as drush
. Replace with php /var/aegir/drush/drush.php
if needed.
Pay attention to these two commands, they require modification to match the name of the site, the alias of the platform, and the name of the backup archive. These commands should each be on one line. You also need to know the platform alias. You can see all the platform aliases available by typing drush sa | grep platform
.
First tell Aegir about the site, which creates a site alias.
drush provision-save @example.com --context_type=site --platform=@platform_d6 --uri=example.com --db_server=@server_localhost --client_name=admin
Then run the provision-deploy:
drush @example.com provision-deploy /var/aegir/tmp/migrate/example.com-2010-11-11.tar.gz
Note: if you receive an error, you might also try adding the site to your new Aegir platform (using the same platform and site name, of course). Then run the provision-deploy again.
3. Verify the Platform
The final step is to import the site into Aegir. This is easily done by logging into the Aegir front-end and verifying the platform, which then will trigger an import task for the new site.
4. Caveats
The provision-save command will assume the database server is the same as the master server and that you are not using any install profile. So if your DB lives in another server or if you are using an install profile like openatrium you will probably need to edit the file created at /var/aegir/.drush/example.com.alias.drushrc.php
5. Setting a Non-Default Install Profile
If you are using a non-default install profile, you may want to specify your install profile by adding --profile=profilemachinename
to the drush provision-save
command above.
Using the example above but using an install profile for Open Atrium, the command would look like:
drush provision-save @example.com --context_type=site --platform=@platform_d6 --profile=openatrium --uri=example.com
Importing a single site manually
If you already have a platform setup on Aegir with EXACTLY the same codebase as your existing site, then you don't need to transfer the entire old codebase - you can just transfer the sites/example.com directory. However, you also need to make sure any dependencies on contrib modules that your site has, are covered on the codebase or Platform that you're importing it into.
In general it may be considered safer/less prone to confusing errors to transfer the entire old codebase into Aegir as a whole Platform, whereby the site will be imported automatically under that Platform. You can then migrate it in Aegir to one of your existing Platforms later. See importing from a complete Drupal platform for this method.
Also, it is much faster to just "deploy" an Aegir backup than go through the manual procedure here, so you should generally follow that procedure instead of the one defined here unless you have a very hairy setup.
To transer a site manually into Aegir, those are the steps you should follow:
1. Create the site database, user and directory
In order to import the site, you need to create a database and database user for the site, along with a directory in the platform.
The simplest way to do this is to create an empty site in the right platform and overwrite it. Through the commandline:
drush provision-save @example.com --context_type=site --platform=@platform_d6 --uri=example.com --db_server=@server_localhost --client_name=admin
drush provision-install @example.com
Through the UI, this can be done simply by creating a site in Create content.
If this is not working for some reason, you can create the mysql user and database manually using the following:
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, LOCK TABLES, CREATE TEMPORARY TABLES ON databasename
.* TO 'databaseuser'@'localhost' IDENTIFIED BY 'password';
CREATE DATABASE databasename
;
2. Transfer the files
You need to copy the sites/example.com
directory from your old server to the new Aegir server.
To do this, first create a tarball of the site on the old server. Within the sites/example.com
directory type:
tar -zcvf example.com.tar.gz .
Then, on the Aegir server, you can create a directory for it in the codebase (platform) you want to put it under:
cd /var/aegir/platforms/drupal-X.Y/sites
mkdir example.com
cd example.com
wget http://example.com/example.com.tar.gz
tar -zxvf example.com.tar.gz ./files ./modules ./libraries ./themes
Instead of using wget you could of course use SCP or FTP to transfer the tarball onto the Aegir server.
In the above we assume that you have created an empty site as directed above, if not, you will also need to extract the settings.php file, so that last command should instead be simply:
tar -zxvf example.com.tar.gz
Then you will need to modify the settings.php file to follow the user and database created in the first step. Again, this is not necessary if you let Aegir create the site for you.
Once the file is unpacked, check the ownership and group of each file by using ls -al
. It should be either 'aegir:aegir' or 'aegir:www-data'. Change it if necessary. In particular, files that may have been uploaded on your site via modules like imagecache, upload, Profile pics etc, may need to have their ownership changed:
sudo chown -R aegir:www-data $platform/sites/*/files
3. Transfer the Database
Make a backup of the database on your old server (Backup and Migrate module is good for this, or you can use phpmyadmin or your favourite MySQL management tool. It's a good idea to truncate the cache, search and watchdog tables first to reduce the size of the database. The "Backup and Migrate" module does this for you.
If you are importing a site from a previous Aegir installation somewhere, such as from a backup tarball, the database.sql will already be within the tarball you unpacked. You can use this to import into a new database on your new server.
You can import the database with drush using the following command:
drush @example.com sqlc < database.sql
4. Verify In Aegir
In the Aegir web interface click on the name of the platform you have added your site to. You can access the list of platforms via the 'Platforms' tab in the primary links menu.
Execute a new Verify task on this platform via the 'Verify' button in the task list.
Aegir will now re-verify the platform. In the process of this, it will discover your new site and spawn a new 'Import' task for the site.
Once it's done this (it may take a couple of cron runs to dispatch these tasks in the queue) the tasks should turn green (if not, see 'Troubleshooting' below).
At this point, presuming you have updated DNS or are overriding DNS via an entry in your /etc/hosts
file to access the site via the new Aegir server, you can visit the site in a browser and check around to see that it has worked. Pay particular attention to any links in node content that pointed to paths referencing /sites/community.aegirproject.org
if your site was not part of a multisite setup. (Drupal has a habit of storing these paths in the database, or they may have been hard-coded into nodes by users).
Aegir will not go through all your database and update all URLs, so some images or links may be broken. This will need to be done manually. Aegir does attempt to update paths stored in tables such as 'files' though.
It's a good idea to clear the caches, and you may need to get imagecache to rebuild its thumbnails if you use it.
Importing a complete Drupal platform
This way of importing sites is often considered the safest whereby you transfer across an entire codebase containing Drupal core, and define it as a Platform first. This makes sure you bring any dependencies, contrib modules etc, in exactly the same version and configuration that is referenced in the database for the site.
1. Transfer the codebase
Create a tarball of the codebase on the old server. Within the drupal root directory type:
tar -zcvf example.com.tar.gz * .htaccess
Note that we have to specify including the .htaccess because it isn't picked up by the * wildcard due to being prefixed with a full stop. It is important to bring along the .htaccess for a platform so that Aegir can consider these settings when it generates Apache configuration for this platform.
Then, on the new server you can create a directory for it in the codebase you want to put it under:
cd /var/aegir/platforms
mkdir your-new-platform
cd your-new-platform
wget http://example.com/example.com.tar.gz
tar -zxvf example.com.tar.gz
Instead of using wget you could of course use SCP or FTP to transfer the tarball onto the Aegir server.
Once the file is unpacked, check the ownership and group of each file by using ls -al
. It should be either 'aegir:aegir' or 'aegir:www-data'. Change it if necessary. We have to assume you have basic knowledge of Linux and POSIX permissions - we can't be responsible for teaching you sysadmin skills :)
In particular, files that may have been uploaded on your site via modules like imagecache, upload, Profile pics etc, may need to have their ownership changed:
sudo chown -R aegir $platform/sites/*/files
If your site previously resided in sites/defau1t you need to move it because Aegir ignores the default directory (it has no way of understanding what URL this would be accessed by, so it is impossible to 'manage' it). Each site needs its own directory with the correct domain in typical Drupal multisite design:
mv sites/defau1t sites/example.com
If you do this, you may need to also update your filepath's stored in the files table by running the following query on your database:
UPDATE files SET filepath = REPLACE(filepath, 'sites/defau1t', 'sites/example.com');
There are many other places in the database that can suffer the same problem, the node_revisions
table for example. If you have phpMyAdmin available you can easily search the entire database for 'sites/defau1t'. Select the database, then click the Search tab. Once you know which tables are affected you can run a variation of the above query to correct these records.
2. Transfer the Database
Note: This step is not necessary if you are moving your site from one directory on your server (e.g., /var/www/html) to a newly created Aegir directory on the same server (e.g., /var/aegir).
Make a backup of the database on your old server (Backup and Migrate module is good for this, or you can use phpmyadmin or your favourite MySQL management tool. It's a good idea to truncate the cache, search and watchdog tables first to reduce the size of the database (Backup and Migrate does this for you).
Aegir does not provide support for importing databases with prefixed tables so it is best to remove prefixes form the database. However, there is work to bring this support in (http://drupal.org/node/483346) and a solution if you want to keep prefixes on your tables (http://drupal.org/node/483346#comment-3113516)
Unlike traditional Drupal projects the database settings cannot be found in settings.php. The database credentials are stored in the Apache vhost config of the associated site. This is a security measure implemented by the Aegir project.
In the vhost directory on your old server, /var/aegir/config/server_master/apache/vhost.d/
, is a vhost file for every site on your server. The contents of the file can be displayed in your terminal with the command cat thenameofthefile
. At the top of the file you will find the credentials needed to create a database on the new server.
On your new server, manually create a new database and upload the .sql file from your backup.
Then create the mysql user that your site accesses the database as, and grant it all permissions on that database except 'GRANT'.
Connect to the database on the old server:
mysql -u root -p
Create the new database:
create database yourdatabasename
Add the user:
CREATE USER 'yourdatabaseuser'@'localhost' identified by 'youruserpassword';
Give the necessary privileges:
GRANT ALL PRIVILEGES ON yourdatabasename.* TO 'yourdatabaseuser'@'localhost';
Import the database:
mysql -u yourdatabaseuser -p -h localhost yourdatabasename < thenameofthedatabase.sql
3. Setup Platform in Aegir
In the Aegir web interface click on 'Create Content' and 'Platform'. This step is often easiest when using the admin-menu on the top of your screen.
Enter the name you want to use for the platform (eg 'My Platform import'), and the path to the platform on the server (eg '/var/aegir/platforms/your-new-platform')
Click Save to submit the form, and Aegir will spawn a Verify task for this platform into the queue.
Once the task is complete this the task should turn green (click through to the homepage to refesh and check. If not, see 'Troubleshooting' below).
4. Import and Verify Sites In Aegir
Upon completion of the above Platform verification task, Aegir will discover your new site and spawn a new 'Import' task for the site.
Once it's done this (it may take a couple of cron runs to dispatch these tasks in the queue) the tasks should turn green (if not, see 'Troubleshooting' below).
At this point, presuming you have updated DNS or are overriding DNS via an entry in your /etc/hosts file to access the site via the new Aegir server, you can visit site in a browser and check around to see that it has worked. Pay particular attention to any links in node content that pointed to paths referencing /sites/community.aegirproject.org
if your site was not part of a multisite setup. (Drupal has a habit of storing these paths in the database, or they may have been hard-coded into nodes by users).
It's a good idea to clear the caches, and you may need to get imagecache to rebuild its thumbnails if you use it.
5. Migrate To Another Platform
Now that you have your sites under Aegir's control you can take advantage of its power, and easily migrate them to another platform. In the event that your sites were on old versions of Drupal core and a new one is available and present as another Platform on your Aegir server, you can use the Migrate process to upgrade those sites onto the new core.
For details on how to Migrate sites, consult the Migrate documentation.
6. Troubleshooting
For those who are migrating existing sites that live with their files in the sites/community.aegirproject.org folder, things can get tricky. However, you can follow this tutorial and successfully get things moved over rather easily.
Setting up a test Aegir site from an existing one
We have already seen how to import sites in an Aegir install, but what if we want to test the "hostmaster" site itself, ie. setup a development or staging environment of an Aegir install itself?
This page explains how. The basic steps are:
1. Installing aegir
The first step should be fairly simple and intuitive if you got that far: you have to install aegir itself on the staging server. You should install the specific version that is currently in production.
Maybe that also involves uninstalling the existing install.
2. Backup the production site
Run a backup on the production site, on the production server. This can be done in the frontend or in the backend using:
drush @hostmaster provision-backup
3. Deploy the production site in staging
Copy the resulting backup to the staging server. Then you should be able to deploy it with the following command:
drush @hostmaster provision-deploy --old_uri=aegir.koumbit.net /home/anarcat/aegir.koumbit.net-20110326.161420.tar.gz --debug
In this case, we use deploy to also rename the site (--old_uri=aegir.koumbit.net
).
4. Tweaking the install for the staging server
The environment on the staging server could be very different from the production server. The server name will be different, maybe there will not be a dedicated mysql server, platforms will not exist, etc.
It is recommended to review the "Servers" page to change the domain names to the staging environment. In Koumbit's case, this means:
- editing the master server to change the hostname and IPs
- renaming the DB server to "localhost" and changing the database credentials
- renaming the main hostmaster site:
drush @hostmaster sqlq "UPDATE node n JOIN hosting_context h ON h.nid = n.nid SET n.title='hostmaster.koumbit.net' WHERE name='hostmaster';"
drush @hostmaster sqlq "UPDATE node_revisions nr JOIN hosting_context h ON h.nid = nr.nid JOIN node n ON n.vid = nr.vid SET nr.title='hostmaster.koumbit.net' WHERE h.name='hostmaster';" - change the sites to point to the good web and db servers:
UPDATE hosting_site SET db_server=3631 WHERE db_server <> 3631;
UPDATE hosting_platform SET web_server = 3 WHERE web_server <> 3;
Note that this will have the added benefit of reverifying all platforms and therefore create them if they are makefile-based.
5. Troubleshooting
I had weird issues with failing to run updatedb on the hostmaster site (while deploying to another alias would work). My workaround was to suspend the process right before it would run drush updatedb, then run updatedb manually, twice:
drush @hostmaster updatedb
drush @hostmaster cc all
drush @hostmaster updatedb
Troubleshooting creating a platform or importing sites
First things to check:
Does the aegir user have proper permissions?
sudo chown aegir -R PLATFORM/sites/yoursite
Is the database properly set up? Try drush sql-cli to see if the site db user has access and if there are tables (try SHOW TABLES).
If something goes wrong with importing a site you might end up with a node for a non-working site. In such a case it can be good to delete the node for this non-working site and verify the underlying platform again. Since there is no direct way to delete such a node from the user interface you have change the URL from node/NUMBER/edit into node/NUMBER/delete to do this.
Remote servers (multiserver)
Aegir supports multiple 'server' entities. These servers have 'services' such as 'Web' or 'Database', and 'service types' which are implementations of that service, such as 'Apache' or 'MySQL'.
Some previous experience in the configuration of Apache and MySQL will help if you want to use remote servers with Aegir. If you haven't had any experience in setting up and maintaining Apache or MySQL before you might want get familiar with the basic concepts first.
1. Remote web servers
1.1. System dependencies
On the remote server, install these packages
apt-get install rsync apache2 php5 php5-cli php5-mysql postfix mysql-client sudo
1.2. Aegir user
Any number of remote web servers may be configured. The remote server needs an aegir user created on the system.
adduser --system --group --home /var/aegir aegir
adduser aegir www-data #make aegir a user of group www-data
1.3. Web server configuration
You'll also need to prepare the web server in the same way you did for the master Aegir server during installation:
a2enmod rewrite
ln -s /var/aegir/config/apache.conf /etc/apache2/conf.d/aegir.conf
Don't restart Apache even when it prompts you. This will be done by the Verify task you'll spawn for the server from the Aegir frontend later.
1.4. Sudoers
Add the aegir user to sudoers so it can restart Apache (e.g. sudo visudo -f /etc/sudoers.d/aegir):
aegir ALL=NOPASSWD: /usr/sbin/apache2ctl
1.5. Login shell
The remote aegir user will also need a login shell, which can be modified using the chsh
command.
chsh -s /bin/sh aegir
1.6. MySQL access
Aegir connects directly to the remote server's MySQL to create databases and users. Therefore, you need to log into MySQL in the remote server as root and create a user with the following command:
GRANT ALL PRIVILEGES ON *.* TO root@aegir_server_ip IDENTIFIED BY 'password' WITH GRANT OPTION;
FLUSH PRIVILEGES;
Where
- 'root' is the username on the Aegir server that will be connecting. Leave 'root';
- 'aegir_server_ip' is the IP address of your Aegir server. For example '123.123.123.123';
- 'password' is the password that will be used by your Aegir 'root' user to connect to the remote server's MySQL to create databases and users. It is suggested to store that password in a secure location. To increase security, it is recommended to choose a password with 6 to 16 alphanumeric and or punctuation characters.
If MySQL returns something like "Query OK", that means the command was successful. For example:
Query OK, 0 rows affected (0.00 sec)
Note the comment about "bind-address = 127.0.0.1" on http://community.aegirproject.org/installing/manual#Database_configuration
When the server is being verified, Aegir will attempt to create a database and grant privileges to a user for that database. If any of these two fails, verify that your MySQL configuration is correct and that there is no firewall blocking your MySQL port.
You can confirm that MySQL is configured correctly on the remote server by manually connecting from the command line on your Aegir machine:
[aegir@aegir_box:~]$ mysql -h <mysql_server_IP> -u root -p
If your MySql database is running on the same server as the Aegir master and is configured with the name "localhost", you may run an issue where the remote web server tries to access MySql via a local socket rather than the hostname. To resolve this, just change the server hostname for the MySql server in Aegir from "localhost" to the hostname of the server.
1.7. SSH keys
SSH public/private keys should be set up so the main Aegir server's aegir user can access remote web aegir users with no passwords required.
Example: on main Aegir server:
ssh-keygen -t rsa
(follow prompts)
Do not use a passphrase when you create your key. Simply press Enter to leave the passphrase blank. Otherwise, SSH will insist that you enter the passphrase everytime you try to SSH using the key. We don't want SSH to present any prompts.
Put the public key's contents into /var/aegir/.ssh/authorized_keys
on the remote web server. The easiest way to do this is to use the ssh-copy-id command.
You should manually login for the first time from your Aegir master server to your remote server as the aegir user, so that the remote web server is added to the known_hosts file in /var/aegir/.ssh on your Aegir master server. Verifying the remote webserver will fail until this has been done.
You can confirm that SSH is set up correctly by manually connecting from your aegirmaster server to your remote server as aegir@ user to aegir@ user:
[aegir@aegirmaster:~]$ ssh aegir@<remoteserver>
[...]
[aegir@<remoteserver>:~]$
There are many, many tutorials online for setting up ssh keys, and various mistakes can be made by inexperienced users such as permissions etc. Aegir isn't a 'Linux beginner's practice tool', so setting these up is really out of the scope of this document and users are encouraged to research this on their own. (For Ubuntu, see this article).
If you have followed the instructions above, and your SSH connection gets closed immediately after you try to connect to the remote server as the aegir user, you may not have given the aegir user a shell correctly. Check the /etc/passwd file on the remote server to make sure that the aegir user has a shell.
1.8. Verify the server
Now you can add a new server node in Aegir, set the hostname and/or IP and set the service type to be 'Apache' (or Apache_SSL if this site is to handle SSL sites).
A verify task will be spawned and added to the Task queue ready for dispatching. During a server verification task, various configurations will be set on the Aegir master server and also synced to the remote web server, restarting Apache.
Now when you add a new Platform node in Aegir, you have the option of setting which web server to host it on. If you are not using a makefile but are downloading a platform manually, you must still do this on the main Aegir server. The contents will then be synced across to the web server.
You don't choose a web server when installing a new site. Because a site depends on a platform, its web server is implied by which platform has been chosen. In other words, all sites on a platform are hosted on the same server. You cannot host two sites on the same platform on different servers.
Remote webserver configuration
Any number of remote web servers may be configured. They need an aegir user and webserver configuration, with the same user name and directory paths. SSH public/private keys should be set up so hostmaster's Aegir user can access remote web Aegir users with no passwords. The above Apache configuration needs to be performed too.
Additionally, the remote web server needs the 'mysql' client binary installed so that Aegir can test that it can successfully connect to the database server. On (at least) Debian-based systems, this can be installed via the 'mysql-client' package.
They will also need a login shell, which can be modified using the chsh
command.
Shell commands as root::
apt-get install mysql-client
adduser --system --group --home /var/aegir aegir
adduser aegir www-data #make aegir a user of group www-data
chsh -s /bin/sh aegir
apt-get install rsync apache2 php5 php5-cli php5-mysql
mkdir /var/aegir/.ssh
cat > /var/aegir/.ssh/authorized_keys <<EOF
ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB...UF aegir@filer01
EOF
chown aegir:aegir /var/aegir/.ssh -R
chmod 750 /var/aegir/.ssh
chmod 640 /var/aegir/.ssh/authorized_keys
a2enmod rewrite
ln -s /var/aegir/config/apache.conf /etc/apache2/conf.d/aegir.conf
cat > /etc/sudoers.d/aegir <<EOF
Defaults:aegir !requiretty
aegir ALL=NOPASSWD: /usr/sbin/apache2ctl
EOF
chmod 440 /etc/sudoers.d/aegir
For clusters, use the aegir-cluster-slave package to operate the above and NFS mounts for you. See also web cluster configuration documentation.
Adding a platform on a remote web server
Especially if you have been using Aegir for a while simply because it handles updates, moving to new releases, cloning, aliases and everything Drupal with such elegance on a single server - you might not be entirely clear on what changes when you make the move to multiple servers. Hopefully this will help.
Understanding the process
Multiple servers are great even if you do not have to manage a server farm. Having a server for development that you are free to break, one for testing that you hope will not break, one for approvals so your clients can go to see how their "real" website will look, and one for production that serves the daily grind of live sites is a wonderful thing. This DTAP (Development, Testing, Acceptance, and Production) structure works and it is surprisingly easy to accomplish under Aegir.
If you are managing a farm, creating massive numbers of servers is quick and easy with Aegir - especially if you use virtual machines like Xen where a server image can be cloned or generated using LVM disks that can be migrated (even while running) to other physical machines.
When it is time break out an Aegir master controller and remote slaves you have to remember that Aegir is the place that it (almost) all happens. Everything is stored in Aegir - including your remote platforms and your sites - and then distributed to the slave servers by Aegir in the Verify process.
Remember too that sites reside on platforms, and platforms reside on servers. You are turning your Aegir install into an Aegir Master as the hub, and the Aeger Remote Slaves as the spokes. To move into this "hub and spoke" architecture, you must build the remote server, tell the Aegir Master about that server and verify it, install a Drupal platform for that server on the Aegir Master and finally tell the Aegir Master about that Drupal install and verify that. You then create the sites on the platform you want based upon the install profile you select for that site. Need more? How about an example:
The mechanics
(1) Build a server image on your network to become your Aegir Slave. This can be anything from a separate headless server to a Virtual Machine image on physical machine with a dozen other Virtual Machines on it. As long as it has its own IP and is capable of running a Drupal image, it will work.
(2) Ensure that your new server image is resolved in your DNS by adding it to your zone files and incrementing the serial number in the zone file. Aegir can manage that for you now to some extent, but you might want to wait until that feature leaves the experimental status. Make sure the entries in /etc/hostname and /etc/hosts all match your zone files (including reverse DNS). Reload/restart the DNS server. Any DNS slaves should automatically get notified and the incremented serial number should make sure they reload automatically. Test the resolution and ping the new Aegir Slave server image from the Aegir Master to Ensure that the IPs all resolved properly.
(2) Follow the steps in the Aegir Handbook page at http://community.aegirproject.org/node/30 carefully. You should be able to use cut and paste if you are using Debian Stable or the like. In any case, you will need those packages. You do not need Aegir on the slave, but you do need the aegir user and that user must have a shell. I prefer the bash shell, but the sh shell in the instructions will do.
(3) You need to have the "passphraseless" ssh working and proven before you continue. A web search on the terms "passphraseless ssh" plus your operating system will generally get you great instructions, but putting those instructions here for every possible combination is beyond the scope of this example. In terms of helpful hints, however, you do not need to have a password for the aegir user on the Aegir Slave. Your access will be this certificate. By simply hitting enter when it asks for the passphrase, you make the certificate passphraseless. You should carefully set the permissions and back these certificates up to a secure location. You MUST log in manually from the Aegir Master to the Aegir Slave the first time or your verify will fail. This is because the system has to add the ssh keys to the known hosts and it will ask about about it before it will allow the connection. Aegir nows nothing about this and will not answer, so the verify will fail. To be absolutely sure it works, after exiting the initial login, you should retry the login to verify it does not ask for a password and then exit the login and try an rsync (which is the process by which the Aegir Master controls the Aegir Slaves). All of these commands should work from the Aegir Master to the new Aegir Slave while logged into the Aegir Master as the aegir user with no request for a password, proving that the systems are exchanging keys (do not proceed until you can do this flawlessly without having to enter a response):
## Start from the Aegir Master as the aegir user
# Log in to the slave from the Aegir Master and exit without any password or reply required
ssh slavename.example.com
exit
# Create a test file for rsync
echo 'rsync test file' > MyRsyncTest
# Send the test file to the remote slave by name using rsync
rsync -azvv /var/aegir/MyRsyncTest aegir@slavename.example.com:/var/aegir
# Log back into the remote slave
ssh slavename.example.com
# Make sure the file is there and readable
cat /var/aegir/MyRsyncTest
# Delete the test file from the remote slave
rm /var/aegir/MyRsyncTest
# Exit the ssh login session on the remote
exit
# Delete the test file on the Aegir Master
rm /var/aegir/MyRsyncTest
(4) Go to Add Content/Server and put in the server name, leave the Database set as None (Your sites will use the Aegir Master database and none is required on the Remote Slave) and under Web select apache or apache ssl (to match the example) then click Save. After the process completes you should see that it found the IP address properly on its own, plugged it in all by itself and then Verify quite nicely. If you look on the Aegir Slave remote server you will see that you now have directories named config and platform.
(5) Now that hard decision for platform choice comes in play. My preference is a Drupal install that is the same release as the Aegir Master server or higher. Since sites are eventually destined to be Drupal 7 one day, and the Aegir system is not Drupal 7 ready as of this writing but heading that direction, making at least one Drupal 7 slave is a good idea so that you can migrate sites to that platform as they are ready. If you are migrating sites off of your existing Aegir and onto slaves, migrating to platforms of the same Drupal version makes that process much easier. In any case, firm up your decision about which Drupal version you want to install on your new Aegir Slave Platform.
(6) Time to create that platform. The big thing here is to do it right the first time and make certain that the name of the platform is unique to any platform on your Aegir Master server. If you look in your /var/aegir directory you will notice that Aegir has its own naming convention. For instance, you should see /var/aegir/hostmaster-0.4-rc1 as the platform for the Aegir 0.4 release candidate 1. You need to consider doing the same. For instance, name your platform something like: /var/aegir/slavename-drupal-version and you will make it easier to distinguish which server the platform resides upon and the drupal version so that you can manage updates and migrations more easily. Make certain that you are not destroying one platform when creating another. Be aware that at some point Aegir started organizing platforms into a separate directory: /var/aegir/platforms and you should change to that structure if you have an older original install. Again, be careful you are not overwriting an existing install when you create the new one. For instance, if you already have a Drupal install in /var/aegir/platforms/drupal-7.14 the code below will attempt to recreate it on top of the existing install. If you are using a Makefile, be sure that the same problem does not occur in that makefile. To create a Drupal 7 Aegir Remote Slave Platform, log into the Aegir Master as the aegir user and run this simple code (of course edit it to match the platform and name you want to create:
# Make sure you are in the proper directory
cd /var/aegir/platforms
# Install Drupal 7.14
drush/drush.php dl drupal-7.14
# Rename the platform to your own naming convention
mv /var/aegir/platforms/drupal-7.14 /var/aegir/platforms/slavename-drupal-7.14
(7) The final step in creating this platform is to actually tell the Aegir Master about it so that it can distribute it (via rsync) to the new Aegir Remote Slave. Simple enough. Just go to Create Content/Platform and the Create Platform screen comes up. Under Name write something descriptive and unique. In the example so far, this will be something like 'slavename Drupal 7.14' or in some way adhere to your chosen naming convention. I use the directory name I created above since that serves two purposes. Under Publish Path you will put the path that holds this Drupal distribution. In the example above that will be /var/aegir/platforms/slavename-drupal-7.14 but the path will be reflected from the Aegir Master to the Aegir Slave and will be the same on both. If you are using a Makefile you need to enter the full path and name of that Makefile here as well, but the creation and use of the Makefile is beyond the scope of this example. Lastly choose the server that this platform will reside upon. When you are finished, double check the screen and then click on Save. The Verify step should execute flawlessly and when it does, your Aegir Remote Slave will be complete and ready to accept sites.
(8) Lather, rinse and repeat for each Aegir Remote Slave.
Creating sites on the remote
The only difference you will notice when creating a new site under Aegir will be the choice of platform. The Create Site screen is quite self explanatory, but the main thing is to select the Install Profile that you require and then the list of Platforms that support that install profile will be populated.
As usual, Aegir makes it all faster and easier than you expect.
Web clusters
Cluster module
The cluster module is a simplified solution for maintaining platforms across multiple web servers. A cluster server node does not require a physical machine to be present on the network. Simply create web servers as usual and create a server node that has "Cluster" selected under the web field set of the server node add form. Select the servers that should be part of the new web cluster by clicking the check box next to each web server in the "Servers" section.
The cluster module will rsync the platforms between each web server and keep them up to date.
Pack module
A Pack consists of a single server node that is specified as "pack" and any number of server nodes specified as "apache" or "nginx". The main difference between the Pack functionality and Web Cluster functionality is that Aegir rsync's configuration (Apache & Nginx) to "all" nodes in the Pack and rysnc's the platform and site code to only the "master" node in the Pack. All other "slave" nodes will see the platform and site code via NFS.
Configuring the Pack server node:
The "pack" server node will not be used by Aegir to physically deploy sites or platforms on, so consider it more of a 'virtual server group' for logistical purposes only. When creating the pack node, you do not need to supply a valid Server hostname or IP address, so choose a naming convention that makes sense in your environment.
Next, select the "pack" radio button option under the web configuration when creating or editing the server node, in order to designate this server node as the "pack" server.
Now select the "Master" server from the list of Master servers. The Master server will be the server node that Aegir rsync's the platform and site to. Typically, you would choose the Aegir server itself as the 'master'.
Finally, select the "Slave" servers from the list of Slave servers. The Slave servers will have the Apache or Nginx config rsync'd to them but not the platform or site code, since those will be available to all Slave servers via the NFS share.
Configuring the Web server nodes (Master and Slaves):
Configure all web server nodes using these instructions. Take care to ensure the 'aegir' user and group on your NFS client machines, have the same UID/GID as that of the NFS server, or else you may run into permissions issues with NFS.
Then mount the files on the remote server.
On the NFS server:
sudo apt-get install nfs-kernel-server
echo "/var/aegir/platforms 10.0.0.0/24(rw,no_subtree_check)" >> /etc/exports
sudo service nfs-kernel-server reload
(Replace the subnet here or add specific /32 hosts as necessary for your environment)
Then on the web servers (Master, if it wasn't also the NFS server, and the Slaves):
sudo apt-get install nfs-client
sudo mount 10.0.0.1:/var/aegir/platforms /var/aegir/platforms
(Replace the NFS server's IP here with that of your master server/Aegir server)
Add this to your fstab on the servers that mount the NFS share, so that the share is mounted on boot:
10.0.0.1:/var/aegir/platforms /var/aegir/platforms nfs rw 0 0
Creating a Platform on a Pack:
When configuring a Platform to be deployed on a Pack, choose the "Pack" server node from the Web server radio group during the Platform node creation. Then you choose this Platform as usual when adding a site, and that site will be served from any servers within the Pack.
Caveats
Relying on an NFS share to serve your entire /var/aegir/platforms can be a single point of failure if the NFS share becomes unavailable. You may want to look into providing some sort of failover for NFS (google for things like DRBD and Heartbeat), or using some other form of redundancy for your NFS (Netapp filer clusters etc)
There have also been reports of issues with MySQL GRANT statements not being provided to all webservers in the pack - especially where the master server is not the Aegir hostmaster server. This issue is ongoing - see the Talk page here or this ticket
Also attached is an example diagram of a Pack cluster known to be functioning in production (with an optional MySQL-MMM cluster out of scope for this documentation), which may help you visualise the Pack and how it can be put together.
Setting Up Varnish & Memcache with Aegir
I had the hardest time finding resources & documentation on how to get this configured properly so I am adding this to the documentation for those of you who may need it.
A couple of notes: I am using Ubuntu 10.04 LTS. I set my apache to port 8082 because there was a conflict on port 8080 on my system. You should make a copy of each file before editing it. Memcache settings are drupal 7.
Varnish
Setting up varnish isn't too hard but it could end up being kind of tedious if you configure something incorrectly & you have alot of sites.
First you install it
sudo apt-get install varnish
Then you have to go to configure it. I use vim.
sudo vim /etc/varnish/default.vcl
These are my settings.
backend default { .host = "127.0.0.1"; .port = "8082"; .connect_timeout = 600s; .first_byte_timeout = 600s; .between_bytes_timeout = 600s; } sub vcl_recv { // Remove has_js and Google Analytics __* cookies. set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", ""); // Remove a ";" prefix, if present. set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", ""); // Remove empty cookies. if (req.http.Cookie ~ "^\s*$") { unset req.http.Cookie; } // Skip the Varnish cache for install, update, and cron if (req.url ~ "install\.php|update\.php|cron\.php") { return (pass); } // Cache all requests by default, overriding the // standard Varnish behavior. // if (req.request == "GET" || req.request == "HEAD") { // return (lookup); // } } sub vcl_hash { if (req.http.Cookie) { set req.hash += req.http.Cookie; } }
Now you have to edit /etc/default/varnish
sudo vim /etc/default/varnish
Here are my settings
# Should we start varnishd at boot? Set to "yes" to enable. START=yes # Maximum number of open files (for ulimit -n) NFILES=$(ulimit -n) #Malloc size depend on your memory server DAEMON_OPTS="-a :80 \ -T localhost:6082 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -s malloc,512M \ -u www-data \ -g www-data"
Now we have to configure apache's listening ports
sudo vim /etc/apache2/ports.conf
change
NameVirtualHost *:80 Listen 80
So that it looks like this
NameVirtualHost *:8082 Listen 8082
Start varnish
services varnish start
Now login to aegir Edit your server & change the port from 80 to 8082
Aegir will re-verify the server, as well as all platforms & sites on the server.
If that doesnt work, you need to go to /var/aegir/config/server_master/apache/vhost.d and edit all of your virtual hosts so that the port shows 8082 instead of 80 then do the same thing with /var/aegir/config/server_master/apache.conf Reverify your server.
Now add this to the local.settings.php of the sites that you want to leverage varnish
<?php
# Tell Drupal it's behind a proxy
$conf['reverse_proxy'] = TRUE;
# Tell Drupal what addresses the proxy server(s) use
$conf['reverse_proxy_addresses'] = array('127.0.0.1','your-ip-address-here');
# Bypass Drupal bootstrap for anonymous users so that Drupal sets max-age > 0
$conf['page_cache_invoke_hooks'] = FALSE;
?>
Memcache
There is hardly any current documentation for setting up memcache on D7, especially not for aegir.
FIrst you install memcache & its dependencies Download & install memcache module into the platform of the site/sites in which you will be using memcache
Then you have to edit
/etc/default/memcached
sudo vim ./etc/init.d/memcached
Here are my settings
#! /bin/sh # PORT=11211 USER=nobody MAXCONN=1024 OPTIONS="" DAEMON=/usr/bin/memcached RETVAL=0 prog="memcached" start_instance() { echo -n $"Starting $prog ($1): " start-stop-daemon --start --quiet --pidfile /var/run/memcached/memcached.$1.pid --exec $DAEMON -- -d -p $PORT -u $USER -m $2 -c $MAXCONN -P /var/run/memcached/memcached.$1.pid $OPTIONS RETVAL=$? echo [ $RETVAL -eq 0 ] && touch /var/lock/memcached.$1 PORT=`expr $PORT + 1` } stop_instance() { echo -n $"Stopping $prog ($1): " start-stop-daemon --stop --quiet --oknodo --pidfile /var/run/memcached/memcached.$1.pid --exec $DAEMON RETVAL=$? echo if [ $RETVAL -eq 0 ] ; then rm -f /var/lock/memcached.$1 rm -f /var/run/memcached/memcached.$1.pid fi } start () { # insure that /var/run/memcached has proper permissions mkdir -p /var/run/memcached if [ "`stat -c %U /var/run/memcached`" != "$USER" ]; then chown $USER /var/run/memcached fi start_instance default 64; start_instance block 16; start_instance content 128; start_instance filter 128; start_instance form 32; start_instance menu 16; start_instance page 8; start_instance update 8; start_instance views 8; start_instance path 8; start_instance field 8; start_instance rules 8; start_instance token 8; start_instance image 8; start_instance apachesolr 16; } stop () { stop_instance default; stop_instance block; stop_instance content; stop_instance filter; stop_instance form; stop_instance menu; stop_instance page; stop_instance update; stop_instance views; stop_instance path; stop_instance field; stop_instance rules; stop_instance token; stop_instance image; stop_instance apachesolr; } restart () { stop start } # See how we were called. case "$1" in start) start ;; stop) stop ;; status) status memcached ;; restart|reload|force-reload) restart ;; *) echo $"Usage: $0 {start|stop|status|restart|reload|force-reload}" exit 1 esac exit $?
Start memcache
sudo services memcached start
If you see errors u may have an issue where a one line got split into two lines.
Check and make sure that none of your lines got broken off.
Place this code in your local.settings.php file inside your site folder
// the path to the core cache file include_once('./includes/cache.inc'); // the path to the memcache cache file include_once('./sites/all/modules/memcache/memcache.inc'); // make MemCacheDrupal the default cache class $conf['cache_default_class'] = 'MemCacheDrupal'; # Key Prefix: edit this for multisite use. $conf['memcache_key_prefix'] = "$_SERVER[db_user]"; $conf['memcache_servers'] = array( '127.0.0.1:11211' => 'default', '127.0.0.1:11212' => 'menu', '127.0.0.1:11213' => 'filter', '127.0.0.1:11214' => 'form', '127.0.0.1:11215' => 'block', '127.0.0.1:11216' => 'update', '127.0.0.1:11217' => 'views', '127.0.0.1:11218' => 'content', '127.0.0.1:11219' => 'apachesolr', '127.0.0.1:11220' => 'path', '127.0.0.1:11221' => 'field', '127.0.0.1:11222' => 'rules', '127.0.0.1:11223' => 'token', '127.0.0.1:11224' => 'image', ); $conf['memcache_bins'] = array( 'cache' => 'default', 'cache_menu' => 'menu', 'cache_filter' => 'filter', 'cache_form' => 'form', 'cache_block' => 'block', 'cache_update' => 'update', 'cache_views' => 'views', 'cache_views_data' => 'views', 'cache_content' => 'content', 'cache_apachesolr' => 'apachesolr', 'cache_path' => 'path', 'cache_field' => 'field', 'cache_rules' => 'rules', 'cache_token' => 'token', 'cache_image' => 'image', );
And that should be basically it.
Working as aegir user
It can be convenient to do stuff as the aegir user. The default shell of the aegir user is /bin/false, so you have to make sure to use something more usable:
sudo su - aegir -s /bin/bash
GNU Screen is a super useful window manager for the console. When launching screen you will usually get errors like
Cannot open your terminal '/dev/pts/6' - please check.
You can get around this by running
script /dev/null
All in all this works nicely:
sudo su aegir -s /bin/bash -c "script -q /dev/null"
It can be convenient to put it in a little script as /usr/local/bin/suaegir
Allowing other users besides root on your system to run commands as aegir
If you wish to allow other users on your system to run commands as the aegir user using sudo, without allowing them to use sudo generally, you can add the following two lines to your /etc/sudoers file using visudo:
User_Alias AEGIRUSERS = comma, separated, list, of, users
AEGIRUSERS ALL = (aegir) ALL
If you want to allow these users to use aegir without entering a password, simply change the second line to this:
AEGIRUSERS ALL = (aegir) NOPASSWD:ALL
Adding apache config files
If you connect to your Hostmaster server and look at ~aegir/config/server_examplecom/apache.conf, you can see that this file includes files from pre.d and post.d directories. The includes are done in the following order:
- pre.d
- vhost.d
- platform.d
- post.d
The earlier files take precendence over the later files for <VirtualHost> blocks. The latter files take precedence over earlier files for <Directory> blocks.
If we wanted to add a phpini directive that can only be set on a PHP_INI_PERDIR-for example, post_max_size-we could create a file called upload.conf and add it either to pre.d or post.d.
Here's a possible upload.conf:
####################################
# Custom Server-wide configuration #
####################################
# our php.ini sets memory limit to 96M
#
# post_max_size should also be less than memory_limit.
# http://us2.php.net/manual/en/ini.core.php#ini.post-max-size
#
php_value post_max_size 90M
php_value upload_max_filesize 85M
If we put upload.conf in post.d, our setting would be applied only if the same setting is not defined higher up the chain by the aegir code. As Antoine states here post.d is mainly useful when migrating in sites. It would also be useful if you wanted to set a server-wide default which could be overridden on by a virtual host or platform config setting.
Using SSL
Introduction
SSL support was significantly improved in Aegir 0.4 alpha9 and subsequent releases have further refined the SSL functionality. Here are the current steps to configure SSL support in Aegir and apply it to your web sites.
Prepare Your Server
- Make sure port 443 is open for SSL traffic.
- From the command line, install SSL software for your web server (e.g. on Debian/Ubuntu you can use
sudo apt-get install openssl
). - Enable SSL support (e.g.
sudo a2enmod ssl
). You will need to restart Apache at this point.
Enable SSL Support in Aegir
- You have to enable SSL support in Aegir as it is off by default. Assuming the URL of your Aegir front end is aegir.example.com, browse to aegir.example.com/admin/hosting/features
- Click on Experimental to reveal experimental features
- Check SSL support
- Click Save configuration
Configure Your Aegir Server
- Click on the Servers tab
- Click on the server that you wish to enable SSL support
- Click Edit to change the server configuration
- Click apache_ssl (this will reveal an additional field: SSL port, which should be already populated with 443).
- You may also have to add an IP address to the IP addresses field. The field is used as an "IP pool" for SSL certificates: IPs are assigned to SSL sites when they are created on a first come, first served basis. (Note that there are known issues with this process, see issue #1126640 for details.)
- Click Save - this will start various tasks beginning with a verify task on the server followed by verify tasks on all platforms that are associated with that server
- If all goes well you will see the following changes in your Aegir file system structure:
a new
/var/aegir/config/ssl.d
directory and a new/var/aegir/config/server_master/ssl.d
directory.
The /var/aegir/config/ssl.d
directory is where you will be able to manipulate SSL keys and certificates, for example by importing commercial SSL certificates or generating a new key. Do not manually edit the /var/aegir/config/server_master/ssl.d
directory as changes to that directory will be overwritten when the server or site are verified.
Configure Your Aegir Site
- You must enable SSL on your sites that are on those platforms associated with the server. Browse to aegir.example.com/hosting/c/example.com
- Click Edit to change the site configuration
- Choose the type of Encryption required and the Encryption key (see the explanatory notes below each option). NOTE:, Alternatively, you may specify a directory under /var/aegir/config/server_master/ssl.d where your own certificate and key is to be stored (see Apache notes below).
- Click Save. Aegir will then generate a certificate and private key for your web site and insert these into a new VirtualHost directive in your vhost file. (This file is typically at /config/server_master/apache/vhost.d/example.com).
- If all goes well the VirtualHost directive will now have these important elements:
<VirtualHost xx.xx.xx.xx:443> # <-- where xx.xx.xx.xx is an IP address dedicated for SSL access to your site and 443 is the port number
....
# Enable SSL handling.
SSLEngine on
SSLCertificateFile /var/aegir/config/server_master/ssl.d/example.com/openssl.crt
SSLCertificateKeyFile /var/aegir/config/server_master/ssl.d/example.com/openssl.key
Now, when you navigate to https://example.com you should see that your site is SSL enabled. It will, however, generate warnings in your browsers because it is a self-signed certificates. See below for how to use commercial certificates to remove this warning.
Notes for Apache users with Commercial Certificate File(s)
If you wish to use your own commercial certificate and key you will need to do the following:
- Follow the directions above, using the "Generate new encryption key" option and using your site's domain name for the "New encryption key". This will create a site directory under
/var/aegir/config/ssl.d/example.com
. With this step, you have created a self-signed certificate, and your site is now configured to use it. - This generated a 2048 bit RSA key for you along with a CSR (Certificate Signing Request). If you prefer to generate your own RSA key, replace the files (
openssl.key
andopenssl.csr
) in the/var/aegir/config/ssl.d/example.com
directory with your RSA key and associated CSR. - Copy and paste the
.csr
file into the form for the issuing Certificate Authority (CA) to create your certificate. - When your certificate has been generated, download the files from the issuing authority and place in your temporary folder on your PC. You may have more than one .crt files, in this case you have a "bundle" or what we call a "certificate chain" that you need to add in aegir (see below).
- Transfer all the files to /var/aegir/config/ssl.d/example.com. Rename the site .crt file to openssl.crt. If you have a certificate chain, install it in
openssl_chain.crt
. You should have at least three files in the directory (openssl.crt
,openssl.key
,openssl.csr
, and optionnallyopenssl_chain.crt
).
Verify your site from aegir's frontend.
You should now be able to access your site via https:// using your commercial certificate.
Notes for Nginx users:
It is recommended to allow Aegir to create a default self-signed certificate and key first, and then replace the contents of both files (not the files itself) with your real key and certificate. Any chained certificates (bundles) should be included in the same file, directly below your own certificate - there is no need for extra files/lines like it is for Apache configuration.
DNS
Debian/Ubuntu
There is preliminary support for DNS in Aegir's backend that allows it to manage zonefiles dynamically when sites are created. There are still a few major bits missing (described in this meta-ticket) so we generally recommend people use the wildcard approach for now.
That said, while perhaps not yet ready for production use, Aegir's DNS functionality can be quite useful for local development environments. If you run Aegir locally, you're probably in the habit of hacking /etc/hosts regularly, or if you're more adventurous, you've set up a local DNS forwarder, such as dnsmasq. With Aegir's DNS feature, you can avoid this hassle, and let Aegir handle it transparently.
While Aegir supports both BIND and dnsmasq, this documentation will focus on the former. The first step is to install BIND. On Debian-based distros, this will look like:
apt-get install bind9
Next we need to allow the aegir user to run rndc (the name server control utility). On recent Debian systems, this will involve editing /etc/sudoers.d/aegir, whereas on older releases this would be /etc/sudoers. Modify the aegir line to read:
aegir ALL=NOPASSWD: /usr/sbin/apache2ctl, /usr/sbin/rndc
Then we need to make sure the bind user has access to read Aegir's files, so we'll add bind to the aegir group:
adduser bind aegir
We're now ready to activate the feature in Aegir's frontend. Go to Hosting >> Features (admin/hosting/features) and select "DNS Support" under the "Experimental" fieldset, and save the configuration.
Finally, we need to activate DNS on the Aegir server. Visit the server node (hosting/c/server_master) and click the edit tab (which should bring you to node/2/edit). Then under "DNS Service", select "Bind" and then save.
This should trigger a verify of the server (and then probably the platforms on it too), after which Aegir will be managing your local BIND server, and it's zonefile(s). If you have installed Aegir in /var/aegir, put this line in the file /etc/bind/named.conf.local:
include "/var/aegir/config/bind.conf";
Aegir is now a full-fledged DNS server that you can use to resolve addresses locally. This can be accomplished on Debian-like systems by editing /etc/resolv.conf to include "nameserver 127.0.0.1". If you're running Aegir in a VM, you can just point this to the address of the virtual machine.
CentOS/RHEL
You need to have a working DNS/bind/named server for AEgir to take control of. What follows is rough documentation to reach that stage. It is currently UNTESTED in a production environment.
yum -y install bind-chroot bind-utils bind-libs bind
chkconfig named on
ln -s /var/named/chroot/var/named/named.conf /etc/named.conf
cat >> /etc/named.conf << EOF
controls {
inet 127.0.0.1 allow {
localnets;
} keys {
rndckey;
};
};
options {
listen-on port 53 { any; };
listen-on-v6 port 53 { ::1; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
memstatistics-file "/var/named/data/named_mem_stats.txt";
// Those options should be used carefully because they disable port
// randomization
// query-source port 53;
// query-source-v6 port 53;
forwarders { 192.168.122.1; };
forward only;
// allow-query { localhost; };
// allow-query-cache { localhost; };
};
logging {
channel default_debug {
file "data/named.run";
severity dynamic;
};
};
view localhost_resolver {
match-clients { any; };
match-destinations { any; };
recursion yes;
include "/etc/named.rfc1912.zones";
// Add local zone definitions here.
include "/var/aegir/config/bind.conf";
};
include "/etc/rndc.key";
EOF
service named start
Additional Notes
Additional documentation can be found in the Provision DNS module: http://drupalcode.org/project/provision.git/blob/HEAD:/dns/NOTES.
DNS Wildcard Configuration
Why run a wildcard DNS server?
A wildcard DNS setup lets you automatically have subdomains for a given domain. For example, say you own the domain widgets.com and you want to setup an unlimited number of subdomains like dev.widgets.com, test.widgets.com, customers.widgets.com, etc.... Typically you would have to set these all up individually. A wildcard DNS can let you bypass a lot of configuration. In a development environment it can let you setup any number of test/development sites very quickly and easily. Drupal developers in particular can leverage Drupal's multisite installation feature to setup lots of sites for development or production very quickly.
Creating a DNS wildcard with hosted DNS
Many of the popular domain registration companies allow you to manage your DNS records through a web interface. If you have access to such a facility, adding a DNS wildcard is easy: you just create an A record called '*', pointing to your server's IP address.
For convenience, here are instructions for some popular hosting providers: GoDaddy, hosting.com (which uses the popular cPanel system) and 123-reg.co.uk. It is likely that your host will have a similar system to one of these.
How to configure DNS wildcards on your own server
Ubuntu/Debian
OS X Snow Leopard
Setting up a Wildcard DNS configuration on Ubuntu/Debian
Overview of Steps on Ubuntu/Debian
I. Install Bind 9 II. Add a zone to /etc/bind/named.conf.local. Check the syntax of named.conf.local with named-checkconf III. Add a zone file & Check the syntax of your zone files for errors with named-checkzone IV. Edit /etc/resolv.conf V. Start up Bind VI. Check setup with dig VII. Troubleshoot if needed
I. Install Bind 9
$ sudo apt-get install bind9
II. Edit /etc/bind/named.conf.local to add a zone.
We need to add a zone to /etc/bind/named.conf.local. A zone is a record for a domain. In this case we'll use a fake domain for development purposes. Our domain can be called anything. Let's call it mydev. (I was really tempted to call the domain bullshit - would have been great for screencasts). So at a shell prompt do:
$ sudo nano /etc/bind/named.conf.local
Add this to the end of the file:
zone "mydev" {
type master;
file "/etc/bind/db.mydev";
};
Save the file with Control - O and enter. Exit from nano with Control X.
Check the syntax of the file as follows:
$ named-checkconf /etc/bind/named.conf.local
If the file is syntactically correct the shell prompt will return nothing. If there is an error - make sure you did not miss a quote or semicolon and recheck the file.
III. Add a zone file & Check the syntax of your zone files for errors with named-checkzone
$ sudo nano /etc/bind/db.mydev
Add this to the file replacing 10.0.2.15 with your IP address
mydev. 86400 IN SOA mydev. hostmaster.mydev. (
20091028 ; serial yyyy-mm-dd
10800; refresh every 15 min
3600; retry every hour
3600000; expire after 1 month +
86400 ); min ttl of 1 day
IN NS mydev.
IN MX 10 mydev.
IN A 10.0.2.15
*.mydev. IN A 10.0.2.15
Save the file with Control - O and enter. Exit from nano with Control X. Check the syntax with the following:
$ named-checkzone mydev /etc/bind/db.mydev
If the syntax of the file is correct you should see something like:
zone mydev/IN: loaded serial 20091028
OK
If not review the file for errors. In particular whenever referring to a domain, it must end in a . , thus mydev.
IV. Edit /etc/resolv.conf
resolv.conf tells applications like browsers where to look for DNS info. By default it may have your ISPs info. We need to tell it to check our local DNS server first then the ISP.
$ sudo nano /etc/resolv.conf
Add/change the following
domain mydev
search mydev
nameserver 127.0.0.1
nameserver (your isp nameserver or open dns)
If you are receiving an IP address via DHCP then you need to edit /etc/dhcp3/dhclient.conf too.
$ sudo nano /etc/dhcp3/dhclient.conf
Look for the lines send host-name, supersede domain-name, and prepend domain-name-servers and set them as follows (and make sure they are uncommented ) :
send host-name "mydev";
supersede domain-name "mydev";
prepend domain-name-servers 127.0.0.1;
sudo dhclient
Whew! Almost there.
V. Start up Bind (or Restart)
$ sudo /etc/init.d/bind9 restart
VI. Check setup with ping and dig
$ ping testing.mydev
PING testing.mydev (10.0.2.15) 56(84) bytes of data.
64 bytes from ubuntu.local (10.0.2.15): icmp_seq=1 ttl=64 time=0.041 ms
64 bytes from ubuntu.local (10.0.2.15): icmp_seq=2 ttl=64 time=0.039 ms
64 bytes from ubuntu.local (10.0.2.15): icmp_seq=3 ttl=64 time=0.0
If ping works then you should be good to go. If not try dig and part V. To stop the ping action press [Ctrl]+[c].
VII. Troubleshoot
I found the section on Troubleshooting Bind in O'Reilly's "Linux System Administration" helpful. Fortunately, this part of the book is viewable on Google books as of this writing (10/29/2009): http://books.google.com/books?id=-jYe2k1p5tIC&lpg=PP1&dq=Linux%20System%... . See page 66
Getting a Wildcard DNS running on OSX Snow Leopard
Note: it took me several tries to get this working. If you are new to DNS be patient with yourself - you'll get it, but may take a few tries.
In this example I will concentrate on setting a development environment with OSX using wildcard DNS
Overview of Steps
I. Edit /etc/named.conf to add a zone.
II. Add a zone file at /var/named/
III. Check the syntax of named.conf and your zone files for errors
IV. Edit /etc/resolv.conf
V. Set your computers network settings to use 127.0.0.1 as a name server
VI. Start up Bind
VII. Check setup with dig
VIII. Reboot if needed
Before we start a few more notes.
I tend to use the nano text editor to edit Unix configuration files you could use Emacs, VI, Textmate, BBEdit or the editor of your choice.
Backup all these files we are editing so you can start over if you mess up. I didn't do this and it added more time to the project. For example to backup /etc/named.conf do:
$ sudo cp /etc/named.conf /etc/named.conf.bck
Last, most of the files we need to edit are owned by root so you will need to use sudo to edit these files. If you get tired of typing sudo you can become root by doing this:
$ sudo -s
Be careful when working as root or using sudo. You can mess up your system so make sure to backup. All example here are run as root.
I. Edit /etc/named.conf to add a zone
We need to edit named.conf to add our zone.
$ nano /etc/named.conf
I called my zone vmdev so I added this to named.conf
zone "vmdev" IN {
type master;
file "db.vmdev";
};
I added this right before the zone 0.0.127.inaddr.apra and saved the file. So we told Bind to look in /var/named/db.vmdev for this zone.
II. Add a zone file at /var/named/
$ nano /var/named/db.vmdev
vmdev. 7200 IN SOA vmdev. root.vmdev. (
2008031801 ; Serial
15 ; Refresh every 15 minutes
3600 ; Retry every hour
3000000 ; Expire after a month+
86400 ) ; Minimum ttl of 1 day
IN NS vmdev.
IN MX 10 vmdev.
IN A 192.168.0.199
*.vmdev. IN A 192.168.0.199
You can just copy this but be sure to change 192.168.0.199 to you Mac's IP address
III. Check the syntax of named.conf and your zone files for errors
Run this to check your named.conf file:
$ named-checkconf /etc/named.conf
If it returns nothing, your named.conf file is at least syntactically correct. If there is an error, then well you have to diagnose and fix the error.
Now run this to check your zone file:
$ named-checkzone vmdev /var/named/db.vmdev
It should return something like this:
zone vmdev/IN: loaded serial 2008031801
OK
If there are errors then diagnose and fix them.
IV. Edit /etc/resolv.conf
$ nano /etc/resolv.conf
#
# Mac OS X Notice
#
# This file is not used by the host name and address resolution
# or the DNS query routing mechanisms used by most processes on
# this Mac OS X system.
#
# This file is automatically generated.
#
domain vmdev.
nameserver 127.0.0.1
nameserver 192.168.0.1
V. Set your computers network settings to use 127.0.0.1 as a name server
Do this at System Preferences -> Network. You may want to use your ISPs Name server as the second name server
VI. Start up Bind
$ launchctl load -w /System/Library/LaunchDaemons/org.isc.named.plist
The -w option tells OSX to enable Bind at startup
VII. Check setup with dig
$ dig faker.vmdev
Should return something like this:
; <<>> DiG 9.6.0-APPLE-P2 <<>> faker.vmdev
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45640
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
;; QUESTION SECTION:
;faker.vmdev. IN A
;; ANSWER SECTION:
faker.vmdev. 7200 IN A 192.168.0.199
;; AUTHORITY SECTION:
vmdev. 7200 IN NS vmdev.
;; ADDITIONAL SECTION:
vmdev. 7200 IN A 192.168.0.199
;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Oct 14 19:28:56 2009
;; MSG SIZE rcvd: 75
The key here is status NOERROR; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45640 If you get an error then check previous steps.
Now try:
$ ping faker.vmdev
should return:
PING test.vmdev (192.168.0.199): 56 data bytes
64 bytes from 192.168.0.199: icmp_seq=0 ttl=64 time=0.059 ms
64 bytes from 192.168.0.199: icmp_seq=1 ttl=64 time=0.087 ms
but if it does not got to VIII.
VIII. Reboot if needed.
I needed to reboot to get everything to take. Whoila! Have a cookie or something.
Hooking into Aegir (site overrides)
Because Aegir is capable of installing, deploying and moving your sites around, it has to manage the various configuration files that keep your sites running.
These configurations include the standard Drupal settings.php for a site, .htaccess overrides for your HTTP server, as well as HTTP 'VirtualHost' or 'vhost' configuration files that tell your HTTP server where your site is.
A caveat of this system is that Aegir regularly re-generates these configuration files to apply new changes that have been made to the sites or platforms, as well as doing so as a safety mechanism to ensure these files are 'sane'.
For example, if you make a modification to a site vhost or settings.php and it breaks your site, running a 'Verify' task on the site should restore the file back to how it was.
However, sometimes it is necessary to add custom configuration or overrides to these files, and you can't do that if Aegir is regularly wiping your changes.
Fortunately, Aegir provides a series of hooks and inclusions for overriding or injecting customisations into these files safely and persistently.
This chapter shows you how each of these hooks or inclusions work.
Injecting into server-wide vhosts
Changing maximum filesize upload is common when setting up sites in Drupal. As described in Overriding site-specific PHP values you can change this value for each site created with Ægir by adding a .drush.inc file in /var/aegir/.drush directory. But wouldn´t it be nice to be able to do it side-wide?
Here below is the same code from ergonlogic demonstrating how to create a domainname.drush.inc file using the domain name as a condition before the code injection.
<?php
function ergonlogiccom_provision_apache_vhost_config($uri, $data) {
if ($uri == "ergonlogic.com") {
drush_log("Overriding PHP file size values. See .drush/ergonlogiccom.drush.inc");
return array("php_value upload_max_filesize 100M", "php_value post_max_size 200M");
}
}
?>
Apparently you can do this side-wide by omitting the IF statement and create a file with a .drush.inc extension.
For instance, you could create a file called global_settings.drush.inc and put in the following code:
<?php
function globalsettings_provision_apache_vhost_config($uri, $data) {
drush_log("Overriding PHP file size values. See .drush/global_settings.drush.inc");
return array("php_value upload_max_filesize 100M", "php_value post_max_size 200M");
}
?>
Make sure you Verify your site after you create the file. Then scroll through the log and find the message you added in the code:
Overriding PHP file size values. See .drush/global_settings.drush.inc
Injecting into platform-wide vhosts (.htaccess)
Using a .htaccess with an Allow Override all directive in Apache can be a major performance killer, because it requires Apache to stat each subdirectory of the codebase looking for overrides in .htaccess.
As a result, Aegir disables the reading of the Drupal .htaccess in the runtime environment.
This does not mean that the .htaccess is not needed. Instead, when you run the Verify task against a Platform, the .htaccess is studied by Aegir and its contents are copied into the platform-wide Apache vhost configuration, typically located in /var/aegir/config/server_master/apache/platform.d
Need to make a modification to the .htaccess? Simple: you can simply edit it in-place as you normally would, but you must re-Verify the platform in Aegir afterward, in order for those new or modified settings to be 'loaded in' to the platform vhost file.
The end result is improved performance for your sites, without losing any functionality, as you can still customise the .htaccess to your liking.
Injecting into site vhosts
Aegir provides some hook functions in its API, one of which allows you to inject extra configuration snippets into your Apache vhost files for sites.
When would I want to do this?
A good example for this is when you may need to inject a custom Rewrite rule that goes beyond what the Aegir Aliases 'redirection' feature provides.
Or, perhaps you have to inject some htpasswd mod_auth password protection for your site, or perhaps a CustomLog definition. The http_basic_auth module uses this. (code)
Typically you'd just add what you need to the vhost file, but the problem is that Aegir manages these vhosts, and on every Verify task, will rewrite the config from a template, blowing away your changes in the process. Ouch!
Fortunately there is a very easy and elegant solution to this problem to save your configurations persistently across Verify tasks and the like, by means of invoking the Provision hook provision_apache_vhost_config()
, or, if you are using Nginx, provision_nginx_vhost_config()
(and just replace the below examples with 'nginx' instead of 'apache' where necessary). Below "mig5" is just an example, you can replace this with anything as drush looks for all *.drush.inc files in ~aegir/.drush
A simple example
In this example I'll inject a simple 'ErrorLog' apache definition into a vhost to save the site error logs to a file.
- Create a file in
~aegir/.drush
calledmig5.drush.inc
. (Choose<any>
prefix for the file name<any>.drush.inc
). - Add this snippet of PHP to the file:
(Choose
<?php
function mig5_provision_apache_vhost_config($uri, $data) {
return "ErrorLog /var/aegir/" . $uri . ".error.log";
}
?><any>
prefix for the function name<any>_provision_apache_vhost_config
). drush cache-clear drush
- Install a site or verify an existing one
Check your site's vhost config file (in /var/aegir/config/server_master/apache/vhost.d/
) and you'll see the line has been injected into the '#Extra configuration' area of the vhost
<VirtualHost *:80>
DocumentRoot /var/aegir/hostmaster-HEAD
ServerName 1.mig5-forge.net
SetEnv db_type mysqli
SetEnv db_name 1mig5forgenet
SetEnv db_user 1mig5forgenet
SetEnv db_passwd X7KzsFhxhp
SetEnv db_host tardis
SetEnv db_port 3306
# Extra configuration from modules:
ErrorLog /var/aegir/1.mig5-forge.net.error.log
# Error handler for Drupal > 4.6.7
<Directory "/var/aegir/hostmaster-HEAD/sites/1.mig5-forge.net/files">
SetHandler This_is_a_Drupal_security_line_do_not_remove
</Directory>
</VirtualHost>
It's that simple! You can see that via the hook, we pass $uri and the drush data to the function, allowing me to abstract the site url so that each site will get its own error log. You could do extra PHP conditionals to ensure certain data only gets inserted into certain sites of a specific name.
To inject multiple lines instead of one, use an array, i.e
<?php
function mig5_provision_apache_vhost_config($uri, $data) {
return array("ErrorLog /var/aegir/" . $uri . ".error.log", "LogLevel warn");
}
?>
The key point of this is that the file ~aegir/.drush/mig5.drush.inc
file will never be touched by Aegir, so you can rest assured your changes will be respected.
If you want to only inject code into a specific site, wrap your code with an if statement, i.e
<?php
function mig5_provision_apache_vhost_config($uri, $data) {
if ($uri === "<site-name-in-aegir>") {
return array("ErrorLog /var/aegir/" . $uri . ".error.log", "LogLevel warn");
}
}
?>
A more advanced example
Managing multiple versions of a production site can be a tricky proposition, even in Aegir. This is particularly true when you want to use the same canonical domain name to allow users to access one such site of your choosing at any given moment. For instance, in Aegir I might have a site named www.example.com (with an alias of example.com,) and a couple of alternates with the site names of test1.example.com and test2.example.com. If I suddenly decide that I want my primary domain to access test1.example.com, Aegir forces me into a tedious process that ultimately results in site downtime – I have to delete or migrate the existing www.example.com site to a new unused site name (or clone it to an unused site name, and delete the original) and then I have to migrate test1.example.com to the vacated site name www.example.com. Alternatively, I could add the www.example.com and example.com aliases to test1.example.com but then my users would just be redirected to test1.example.com.
Given the way Aegir manages aliases, it would actually be easier to make this switch if the desired address was never used as a site name to begin with. Instead of having the site name www.example.com alongside of the two other test sites, we might have live.example.com, with in-use aliases of www.example.com and example.com, and a rewrite instruction in the vhost that rewrites live.example.com to www.example.com. If we could easily change the rewrite instructions in that vhost, all we would need to do is move the aliases from one site to the next in the GUI (with the requisite verify tasks executed on each site in question) in order to quickly change which site was being loaded at any given moment by www.example.com. For example, if I wanted to move to test2.example.com, I would remove the www.example.com and example.com aliases from live.example.com and verify, add those aliases to test2.example.com and verify, and change my vhost so that test2 .example.com is rewritten as www.example.com. Since as previously mentioned Aegir overwrites changes to the vhost during the verify process, the solution is to use the provision_apache_vhost_config hook in a drush.inc file to selectively add the rewrite information to the vhost when verifying the site that we want our canonical domain to refer to.
<?php
function aliasredirects_provision_apache_vhost_config($uri, $data) {
// the uri to check here is the name of the site in Aegir
if ($uri === "test2.example.com") {
$rval[] = "";
$rval[] = "# Forces redirect to one uri";
$rval[] = "RewriteEngine On";
// if the host is not example.com
$rval[] = "RewriteCond %{HTTP_HOST} !^example.com$ [NC]";
// rewrite to example.com with a 301 redirect
$rval[] = "RewriteRule ^(.*)$ http://example.com$1 [R=301,L]";
$rval[] = "";
return $rval;
}
}
?>
Overriding site-specific PHP values
Sometimes it is useful to override certain PHP values on a per-site basis, but changes to php.ini
are generally server-wide. Depending on the value you want to override, a couple of options present themselves.
First, let's consider where PHP values can be changed. http://www.php.net/manual/en/ini.list.php lists php.ini
directives, and under the "Changeable" column, indicates where a configuration setting may be set. If your value shows either PHP_INI_USER
or PHP_INI_ALL
, then the easiest way to change this value would be using ini_set()
in a local.settings.php
:
<?php
@ini_set('memory_limit', '128M');
?>
On the other hand, if the changes mode is either PHP_INI_PERDIR
or PHP_INI_SYSTEM
, php_ini()
won't work. In this case, the solution is to inject the value into the vhost. Since vhosts are managed by Aegir, manually adding an override to /var/aegir/config/server_master/apache/vhost.d/<uri>
would get blown away the next time that the site is verified.
As described in Injecting into site vhosts, we can inject values into vhosts using a Drush hook. For example, to raise the file upload size limit on http://ergonlogic.com, I added the following code in /var/aegir/.drush/ergonlogiccom.drush.inc
:
<?php
function ergonlogiccom_provision_apache_vhost_config($uri, $data) {
if ($uri == "ergonlogic.com") {
drush_log("Overriding PHP file size values. See .drush/ergonlogiccom.drush.inc");
return array("php_value upload_max_filesize 100M", "php_value post_max_size 200M");
}
}
?>
This results in the insertion of the following lines into /var/aegir/config/server_master/apache/vhost.d/ergonlogic.com
:
php_value upload_max_filesize 100M
php_value post_max_size 200M
Also, in the verify task log I get the following informative message:
Overriding PHP file size values. See .drush/ergonlogiccom.drush.inc
One challenge this technique may present is inspecting the values of the parameters passed into this function. It appears that the Hostmaster site doesn't get bootstrapped, and so common debugging tools (such as devel.module's dd()) aren't available. However drush_log() is, and when called, will push arbitrary messages back into your Aegir site's verify task log.
So sticking the following into the function above can help:
<?php
drush_log("$uri: " . print_r($uri));
?>
Injecting into settings.php
Introduction
Every web site in an Aegir environment has a Drupal configuration file settings.php
in /sites/example.com directory. Web administrators often need to make changes to this file; however, the Aegir system also manages this file and any manual customizations will be lost when a site is verified.
Fortunately, there are two mechanisms to ensure that your customizations can be preserved. If you look in the bottom of an Aegir settings.php
file you will see references to two files local.settings.php
and global.inc
.
<?php
# Additional host wide configuration settings. Useful for safely specifying configuration settings.
if (file_exists('/var/aegir/config/includes/global.inc')) {
include_once('/var/aegir/config/includes/global.inc');
}
# Additional site configuration settings. Allows to override global settings.
if (file_exists('/var/aegir/example-platform/sites/example.com/local.settings.php')) {
include_once('/var/aegir/example-platform/sites/example.com/local.settings.php');
}
?>
If these files exist they are loaded at run time by Drupal. As you can probably surmise from the paths to these files, local.settings.php
is for site-specific customizations and global.inc
is for Aegir-wide customization.
Let's look at these files in more detail. We'll use customization of user session cookies as an example. If you look at the settings.php
file generated by Aegir you see that it sets more conservative php settings for cookies (@ini_set('session.cookie_lifetime', 0);
i.e. cookies expire immediately) than are in the default.settings.php
packaged with Drupal (@ini_set('session.cookie_lifetime', 2000000);
i.e. 2 million seconds, which is just over 23 days).
Site-specific Customization
The local.settings.php
file by default does not exist in a new Aegir site installation so you have to create it. Continuing with our example of user session cookies, let's override the Aegir default.
<?php
# site-specific Drupal customization
# override Aegir-generated cookie policy for sites - set cookies to expire after a week (604,800 seconds)
@ini_set('session.cookie_lifetime', 604800);
?>
Note that because local.settings.php
is included after the variables are set in the main settings.php
it's customizations takes precedence.
Now, whenever you clone a site or migrate it between platforms, Aegir moves a copy of local.settings.php
as well.
Using drush_hook_provision_drupal_config
You can also use the API to add module-specific site configurations with hook_provision_drupal_config
<?php
* Append PHP code to Drupal's settings.php file.
*
* To use templating, return an include statement for the template.
*
* @param $uri
* URI for the site.
* @param $data
* Associative array of data from provisionConfig_drupal_settings::data.
*
* @return
* Lines to add to the site's settings.php file.
*
* @see provisionConfig_drupal_settings
*/
function drush_hook_provision_drupal_config($uri, $data, $config) {
return '$conf[\'reverse_proxy\'] = TRUE;';
}
?>
For example it could look like this
<?php
function drupalwiki_provision_drupal_config($uri, $data, $config) {
$extra = drush_get_option('site_extra_settings', '');
// remove window CR
$extra = str_replace("\r",'',$extra);
return $extra;
}
?>
That is used to add the site settings added by the UI in the hosting backend implemented in https://github.com/EugenMayer/hosting_site_settings
Aegir-wide Customization
In some situations you may want to implement the same configuration settings on all your Aegir sites. This is where global.inc
comes in. Note that global.inc
is now included in settings.php
before local.settings.php
, so that Aegir system administrators no longer retain the ability to override configuration changes in local.settings.php
, but instead it is possible to override global settings per site (read why this has been changed: http://drupal.org/node/1044938 - note: this change is available since 0.4-rc1 release).
For example, say the system administrator wanted to limit users' session lifetimes to a maximum of one day they could create a global.inc
as follows:
<?php
# Aegir-wide Drupal customization
# override Aegir-generated cookie policy for all sites - set cookies to expire after a day (86,400 seconds)
@ini_set('session.cookie_lifetime', 86400);
?>
You can even set more granular policy within global.inc
(however it makes more sense to keep site-specific overrides in the local.settings.php):
<?php
# Aegir-wide Drupal customization
# override Aegir-generated cookie policy for all sites - set cookies to expire after a day (86,400 seconds)
@ini_set('session.cookie_lifetime', 86400);
# Make the aegir front-end server more secure by expiring cookies immediately
if (preg_match("/hostmaster/", $conf['install_profile'])) {
# set cookies to expire immediately on hostmaster
@ini_set('session.cookie_lifetime', 0);
}
?>
If you are using Aegir to manage multiple remote webservers, you will need to run the Verify task on the webserver in order to push global.inc to the remote machine.
Note: on some Aegir installations (e.g. alpha14) the permissions on /var/aegir/config
are insufficent for Apache to properly include the global.inc
file. If you find that the configuration settings in global.inc
are being ignored, change the permissions with chmod a+x /var/aegir/config
. This only has to be applied to the config directory as the sub directory and file permissions should be correct.
If you have other configuration examples, create a wiki and reference it here.
Injecting into drushrc.php
The drushrc.php
file can be changed in two ways:
- by changing the templates used for the Drushrc context (see the hook_provision_config_load_templates() hook)
- by creating a
local.drushrc.php
file in the site-specific folder (e.g.sites/example.com/local.drushrc.php
This is a stub. You can put more stuff here and it would be prettier.
Contributed Modules
1. Aegir 7.x-3.x
Contrib modules for the upcomming version are listed on a sub page
2. Extensions to Hostmaster (frontend)
2.1. Verified in Aegir 2.x
- Aegir HTTP basic authentication (now included in hosting_tasks_extra)
- This module allows you to protect sites with passwords.
- Hosting server titles
- This is a really simple module that allows you to change the names of servers that are displayed in lists in the Hosting frontend when choosing what server to use for something.
- Hosting site Backup Manager
- Add a backups tab to a site node. It extends the default backup functionality with the ability to download a backup over HTTP.
- Now also includes the functionality from Hosting backup garbage collection and Hosting backup queue
- Hosting site Git
- This is a simple module for the Aegir project that adds a 'Git pull', 'Git checkout' and 'Git clone' task for sites.
- Hosting platform Git
- Provides integration with Aegir that allows provisioning a platform from a git repository (also allows specifying the branch or tag name)
- Hosting tasks extra
- This module extends Aegir hostmaster (and drush/provision) with some additional tasks. Such as: cache-clear and registry-rebuild.
- Now also includes the functionality from Aegir HTTP basic authentication
- Hosting database login
- Provides a link to login into a PHPMyAdmin instance (or other db management web software) configured for the current site database server. Requires dblogin to be installed on the site.
- Aegir Piwik integration
- Makes it easy to connect an Aegir instance and a Piwik instance. When you create a site in Aegir, the site is registered in Piwik. When you verify the site in Aegir, new site aliases (if any) get added to Piwik. Also adds system variables to settings.php in order to integrate properly with the Piwik module.
Note: There's a good chance that other Aegir 1.x contrib modules also work on 2.x
2.2. Aegir 1.x
- Aegir Ubercart Integration
- This module allows you to sell an Ægir client, with quotas and access to specific platforms, through Ubercart . It automates a number of set-up tasks from the moment the order ...
- Aegir HTTP basic authentication
- This module allows you to protect sites with passwords.
- Aegir HTTP basic LDAP authentication
- A fork of the above to allow you to specify an LDAP server and Require (ldap filter) string for authentication of 'sites'.
- Aegir feeds
- This is a very basic module that provides feeds integration with Aegir . Features Provides mapping targets for Aegir site nodes, allowing you to create new sites from a feed or file upload.
- Aegir Piwik integration
- Makes it easy to connect an Aegir instance and a Piwik instance. When you create a site in Aegir, the site is registered in Piwik. When you verify the site in Aegir, new site aliases (if any) get added to Piwik. Also adds system variables to settings.php in order to integrate properly with the Piwik module.
- Aegir Rules
- Aegir Rules provides Rules integration for the Aegir.
- Aegir Drush Make Platform working-copy
- This feature allows you to specify 'working-copy' when adding a platform using a Drush makefile. Working-copy means your .git subdirectories are preserved in the platform, which may be useful for development purposes.
- Aegir Subfolders
- Support for 'example.com/subsite1' style sites in Aegir (experimental as of July 2012)
- Hosting Platform Pathauto
- This is an add-on module for the Aegir hosting system: http://aegirproject.org/ This module simply makes a little time-saving tweak when creating a new platform: ...
- Hosting queue runner
- This module allows the Hosting tasks queue for the Aegir project to be 'daemonized' so that tasks are run as soon as possible instead of waiting for a cron run. This makes Aegir appear much more responsive. This module was inspired by the Waiting queue project. Requirements You will need ...
- Hosting Reinstall
- The Hosting Reinstall module provides a frontend for the Provision Reinstall module so that you can reinstall any sites via Hostmaster (Aegir).
- Hosting backup garbage collection
- This is an add-on module for the Aegir hosting system: http://aegirproject.org Introduction This module allows you to specify how long Aegir should retain backups for. You can specify, ...
- Hosting backup queue
- This is a simple module for the Aegir project that allows scheduling of backups. E.g. backup sites every hour, day or month. The module allows for the general settings to be applied to all sites, ...
- Hosting site Backup Manager
- Add a backups tab to a site node. It extends the default backup functionality with the ability to download a backup over HTTP.
- Hosting site Git
- This is a simple module for the Aegir project that adds a 'Git pull', 'Git checkout' and 'Git clone' task for sites.
- Hosting upload
- This module allows administrators to add platforms to aegir and extend platforms and sites with modules, themes, libraries without the need of ssh access.
- Hosting Profile Roles
- Hosting Profile Roles extends Aegir to enable more control over what role(s) a client is assigned during site install. Since, by default, Aegir client's are given UID1 during site installs, this module also allows UID1 to be assigned to another user.
- Hosting server titles
- This is a really simple module that allows you to change the names of servers that are displayed in lists in the Hosting frontend when choosing what server to use for something.
- Services API Support
- Integrates the Services API framework into the Hostmaster suite of tools. It allows Aegir managers the ability to build Services API connections over any of the available servers for Services API.
- Hosting Advanced Cron
- This module allows an administrator to set the default cron interval globally and to override that interval for individual sites.
- Hosting Features
- This module provides a Task to Hostmaster that provides integration with a site's Features. (In beta as of June 1,2012)
- Hosting Notifications
- This is a module that integrates with the Notifications framework. This allows you to receive notifications about task execution in various formats.
- Hosting Task Garbage Collection (merged into Aegir 7.x-3.x)
- This module will delete rows in the {hosting_task} and {hosting_task_log} tables. (In alpha as of June 1,2012)
- Hosting Remote Import
- This Drupal module provides a UI for fetching sites from remote Aegir servers. (In development as of June 1, 2012)
- Hosting Varnish
- Allows the Varnish cache for Aegir-managed sites to be purged via the user interface. (In alpha as of June 1,2012)
- Hosting CivicCRM Cron
- This module provides the ability to trigger CiviCRM cron jobs on sites hosted on the Aegir Hosting System.
- Hosting tasks extra
- This module extends Aegir hostmaster (and drush/provision) with some additional tasks. Such as: cache-clear and registry-rebuild.
- Hosting Filemanager
- This module provides a web file browser for viewing files under sites and platforms. (In beta as of August 29,2012)
- Hosting Signup Settings
- This module is dependent on hosting_signup module in the hosting package of Aegir. It allows the Signup form at /hosting/signup to be configured. (Sandbox project, October 22, 2012)
- Hosting Client Roles
- Client Roles module integrates Aegir user roles into hosting_products as product feature. With this module you can assign Aegir user roles to Ubercart products, and sell them as hosting products. This module is part of the "Hosting & Ubercart Integration" package, and depends on uc_hosting and uc_hosting_products modules
- Hosting Solr
- Works with Provision Solr to give Aegir the ability to manage Solr servers and and gives sites a Solr database.
3. Extensions to Provision (backend)
- Provision boost support
- the Aegir project and therefore depends on the provision backend and expects the Aegir environment to be functional. Quickstart Download the provision_boost code into your /var/aegir/.drush/ Download ...
- Provision ACL support
- This drush command aims to provide preliminary ACL support for the Aegir backend . The objective here is to allow administrators to set more granular access policies on files managed by Aegir and resolve more hairy access permissions requirements than what is usually allowed by the Aegir core. ...
- Project Status
- Project status is an extension for drush which takes a list of drupal platforms as arguments and returns a list of modules which are not in use by any site within these platforms. It is also optionally able to list modules which are use, by platform and site.
- Provision CiviCRM
- A Drush module to automatically setup Drupal instances with the CiviCRM Constituent Relationship Management solution. It is specifically aimed towards the Aegir project.
- Provision Backup Platform
- This hooks into deleting a platform and creates a simple tar backup.
- kPlatforms
- Koumbit's "standard Drupal platforms", a set of makefiles to build and maintain development and production platforms easily.
- Provision Git
- Provision Git is a backend Drush module. It provides 4 drush commands. (In alpha as of June 1,2012)
- Provision Merge Log
- This Aegir extension allows you to automate fetching and merging logs from multiple servers part of the same cluster in a single logfile for later analysis with various analysis tools
- Provision CDN
- This is a simple module and drush script for that allows you to enable CDN support per site in Aegir. (In development as of June 1,2012)
- Provision profile settings
- Incredibly simple extension for provision that allows install profiles to easily inject settings into settings.php for provisioned sites.
- Provision New Relic
- This is a simple extension that exposes all sites in New Relic, so you can monitor each sites performance.
- Provision Solr
- Works with Hosting Solr to give Aegir the ability to manage Solr servers and and gives sites a Solr database.
4. Themes
- Koumbit has provided it's custom Aegir theme as a sample subtheme of Eldir.
- Saeven, a clone of the core theme Seven, based on Omega 4 or 5, clean, simple and responsive.
5. Puppet modules and Chef cookbooks
Those projects allow you to manage Aegir instance(s) through Puppet. Chef recipes also welcome, we don't discriminate.
6. Others
- Aegir up
- Aegir-up is a Drush extension that deploys a local instance of the Aegir Hosting System atop Vagrant and Virtualbox, for development and testing purposes.
- DevShop
- DevShop is a Drupal Development Environment Manager built on Aegir.
-
DevShop creates Aegir platforms and sites automatically from Git URLs. It tracks multiple Projects and allows multiple environments to be created for each Project, such as dev, test, and live. It provides tools to Pull Code, Sync Data, Commit Features, and Run Tests on these environments, and provides a dashboard with useful links and information for developers.
7. Your module here?
Developers: Please add your contributed module here, and feel free to put install instructions or other documentation in a child page. If you are looking for ways of extending Aegir or have already done so and want to document the internals of your module, head over to the extending Aegir page.
Aegir Services
Aegir Services integrates the Services API framework into the Hostmaster suite of tools. It allows Aegir managers the ability to build Services API connections over any of the available servers for Services API.
Using Services 3.x we are aiming at full CRUD support for clients, tasks and sites, as well as read access to profiles, platforms, and eventually, packages.
The 2.x branch of Aegir Ubercart Integration will take advantage of this support to provide a remote storefront functionality.
Please report issues and request support via the issue queue on Drupal.org
Installation
- Install Aegir
- Install the dev version of services 3.x, including at least one server and, ideally, an authentication server.
- Install this module.
- Configure your endpoint, it's resources and authentication, normally.
- Optionally, test your rest server using the included bash script,
hosting_services.rest_test.sh
Contribs for Aegir 7.x-3.x
1. Extensions to Hostmaster (frontend)
- Hosting Git
- This is a simple module for the Aegir project that adds a 'Git pull', 'Git checkout' and 'Git clone' task for sites.
- The successor of Hosting site Git & Hosting platform Git
- Hosting Logs
- This is a simple module for the Aegir project that adds a 'Logs' tab to sites and platforms. Showing Apache error, Git commit and watchdog logs.
- Hosting Platform Pathauto
- This is an add-on module for the Aegir hosting system: http://aegirproject.org/ This module simply makes a little time-saving tweak when creating a new platform: ...
- Hosting site make
- Allows a site to have its modules built from a makefile in the sites directory.
- Hosting tasks extra
- This module extends Aegir hostmaster (and drush/provision) with some additional tasks. Such as: cache-clear and registry-rebuild.
- Now also includes the functionality from Aegir HTTP basic authentication
2. Extensions to Provision (backend)
Starting from Aegir 7.x-3.x the Drush component can be included in a 'drush' directory in the same git repository as the hosting module.
Therefore this list will be shorter then for previous versions.
- Provision STS
- Adds the Strict Transport Security header to hosts that require SSL.
3. Themes
4. Puppet modules and Chef cookbooks
Those projects allow you to manage Aegir instance(s) through Puppet. Chef recipes also welcome, we don't discriminate.
5. Others
- Aegir up
- Aegir-up is a Drush extension that deploys a local instance of the Aegir Hosting System atop Vagrant and Virtualbox, for development and testing purposes.
- DevShop
- DevShop is a Drupal Development Environment Manager built on Aegir.
-
DevShop creates Aegir platforms and sites automatically from Git URLs. It tracks multiple Projects and allows multiple environments to be created for each Project, such as dev, test, and live. It provides tools to Pull Code, Sync Data, Commit Features, and Run Tests on these environments, and provides a dashboard with useful links and information for developers.
- Aegir Pathologic Files
- A tiny Drupal Module to simplify file paths in content. This helps prevent broken images when the site directory name changes. Requires an apache rewrite rule to point /files to /sites/example.com/files, which Aegir provides by default.
- This module is not for hostmaster, but for the sites hosted under Aegir.
6. Your module here?
Developers: Please add your contributed module here, and feel free to put install instructions or other documentation in a child page. If you are looking for ways of extending Aegir or have already done so and want to document the internals of your module, head over to the extending Aegir page.
DevShop Provision
Homepage: http://drupal.org/project/devshop
DevShop Provision
Drupal development infrastructure made easy.
This module provides the backend commands needed to deploy and manage sites using the DevShop git and features based development workflow.
About DevShop
The goals of DevShop are...
- to simplify management of multiple environments for multiple Drupal projects.
- to provide web-based tools that streamline the Drupal site building workflow.
- to provide a common, open-source infrastructure for Drupal development shops.
Installation
To install devshop_provision, simply use drush:
$ drush dl devshop_provision
Or, you can download the source code to any available drush commands folder, such as /usr/share/drush or ~/.drush
Usage
DevShop Provision works by providing a new Provision class called "Project".
To start, you must have a Project drush alias. Using the DevShop+Hostmaster system will make this much easier, but if you wish to only use the backend, you can create a project alias with provision-save, for project NAME:
$ drush provision-save project_NAME --context_type=project --code_path=/var/aegir/projects/NAME --git_url=http://git.url/to/repo.git --base_url=NAME.server.com
$ drush provision-save project_NAME --project_name=NAME --context_type=project --code_path=/var/aegir/projects/NAME --git_url=git@github.com:devudo/drupal-flat.git --base_url=NAME.server.com
Commands
DevShop contains a set of features that make Drupal site building within a version-controlled code workflow quick and easy.
Currently you must create new platforms within a project in hostmaster. Once created, you can execute the following commands:
1. Pull Code
$ drush @project_NAME provision-devshop-pull ENVIRONMENT
This task runs on the dev platform for this project. It runs git pull, and optionally runs new updates, reverts features, and clears caches. It is used to keep the dev server up to date on every commit via the devshop_pull module, and can be used as the deployment task.
- Git Pull the code for your site's platform.
- Then, all optionally:
- Run update.php.
- Revert all Features modules
- Clear caches
2. Commit Features
$ drush @project_NAME provision-devshop-commit ENVIRONMENT --message="My Commit"
This task integrates with Features.module to make it very easy to commit your changes to your features.
- Calls drush features-update-all
- Commits the result, with a part automated and part customized commit message.
- (Optionally) pushes the commits.
- (Optionally) force-reverts after a commit.
3. Sync Content
$ drush @project_NAME provision-devshop-sync SOURCE_ENVIRONMENT DESTINATION_ENVIRONMENT
This task makes it easy to syncronize the database and files down from other environments within the project.
WARNING: This will DESTROY the destination site's database!
This task:
- (optionally) Pulls code
- Drops the @destination database.
- Creates an SQL dump from @source.
- Copies the SQL dump to the local system (if @source is a remote).
- Imports the SQL dump into @destination database.
- (optionally) Runs update.php.
- (optionally) Runs features-revert-all.
- (optionally) Clears all caches.
Commands
$ drush --filter=devshop_provision
All commands in devshop_provision: (devshop_provision)
provision-devshop-co Export the site's Features and commit the result.
mmit (pdc)
provision-devshop-pu Pull & verify platform code and (optionally) run update.php, clear cache, and revert features.
ll (pdp)
provision-devshop-sy Sync database (and files, coming soon) from a chosen source site.
nc (pds)
provision-devshop-te Run a group of SimpleTest tests.
st (pdt)
$ drush provision-devshop-commit --help
Export the site's Features and commit the result.
Examples:
drush @project_mysite Recreates and Commits all features from the dev environment of @project_mysite with an additional commit
provision-devshop-commit dev message.
--message="changed some settings"
Arguments:
environment The name of the environment to commit from.
Options:
--message Add a commit message
--revert Force revert all features after exporting and committing.
Aliases: pdc
$ drush provision-devshop-pull --help
Pull & verify platform code and (optionally) run update.php, clear cache, and revert features.
Examples:
drush @project_mysite Triggers a git pull and a clear cache command on @project_mysite's dev and test environments.
provision-devshop-pull dev test --cache
Arguments:
environments A list of environment names to Pull Code to.
Options:
--update Run update.php after code pull.
--revert Revert all features after code pull.
--cache Clear all caches after code pull.
Aliases: pdp
$ drush provision-devshop-sync --help
Sync database (and files, coming soon) from a chosen source site.
Examples:
drush @project_mysite Syncs the database from for project_mysite's live to project_mysite's dev server.
provision-devshop-sync live dev
Arguments:
from Environment to sync from.
to Environment to sync to.
Options:
--update Run update.php after content sync.
--revert Revert all features after content sync.
--cache Clear all caches after content sync.
--files Sync site files.
Aliases: pds
$ drush provision-devshop-test --help
Run a group of SimpleTest tests.
Arguments:
environment The name of the environment to run tests on.
Options:
--tests-to-run The list of tests to run.
--sync-from-live Sync contents from Live before running tests.
Aliases: pdt
Provision ACL
This page documents the Provision ACL extension to Aegir which allows more granular access control over your sites files and directories.
1. Install instructions
First, you'll need a running Aegir install (1.0-rc3 or later), see http://community.aegirproject.org/installing. Most (if not all) of these commands will have to be run as root (or using sudo, etc.)
1.1. Download and install provision
drush dl provisionacl-6.x
1.2. Enable ACLs on your filesystem
mount -o remount,acl /
Here we assume everything is under the root (/
) filesystem here, otherwise run this command for every filesystem Aegir will work on (e.g. /srv
, /var
or /home
).
You also need to edit your /etc/fstab
for this configuration to survive reboots.
1.3. Install ACL support package
apt-get install acl
1.4. Create a UNIX group
In this case we choose a group called "devs" but you can choose another name.
groupadd devs
1.5. Add users to the group
Add one or more UNIX users that you want to give access to that group. For an existing user (socrates32), this would look like:
usermod -a -G devs socrates32
For a new user (ergonlogic), this would look like:
useradd -G devs ergonlogic
1.6. Create a client
Create a client (should be called "devs" for this example) in the frontend at /node/add/client.
1.7. Create a site
Create a site for the client in the frontend at /node/add/site.
2. What it does
When the site is installed, members of the "devs" group will be able to write to the sites' directories (e.g. upload files and modules) and run drush commands on the site (yes, including site aliases, although see caveats below).
This works also for existing sites; make sure you create a group matching the internal name of the existing client and reverify the site.
3. LDAP integration
Provisionacl supports LDAP groups as well. Ensure that an LDAP client is running and that the 'aegir' user can see the LDAP-provided groups:
getent groups
You may need to restart the Name Service Cache Daemon (nscd):
/etc/init.d/nscd restart
4. API - how to add ACL support to your Aegir extension
To change ACLs on files, you should use something like this:
if (function_exists('provisionacl_set_acl')) {
provisionacl_files_acls(d()->site_path . '/mysettings.php');
}
You can optionnally pass a group as an argument, but it will guess that from the client name of the site. Also note that this will raise a drush error if setfacl fails, but just set a warning if the group doesn't exist.
5. Caveats (ie. what it does not)
Giving shell access to users in Aegir is still insecure, see this upstream issue: #762138.
We aim to refactor this into the Aegir core in 2.x, but in the meantime this should provide a good workaround for the limitations of the existing permission system.
This will only work in 1.0 and above, as it needs the "client_name" field to be populated.
You will need to change your $HOME variable for aliases to work, because of this bug in drush: #1104438. Example:
env HOME=/var/aegir drush @hostmaster cc all
See also this post for context and design.
6. Debugging
If for some reason you have lost the ACLs on the directories and you need to restore them, use the following commands, which is basically what the module does:
cd sites/example.com
setfacl -R -m user:aegir:rwx .
setfacl -R -m default:user:aegir:rwx .
setfacl -R -m group:www-data:rwx .
setfacl -R -m d:group:www-data:rwx .
setfacl -R -m group:cl-group:rwx .
setfacl -R -m default:group:cl-admin:rwx .
7. Notes
This page is the reference documentation for the Provision ACL module page on Drupal.org - keep this in mind when editing please.
Ubercart Integration
This module provides several features to ubercart products on an Aegir platform.
Requirements
- Hostmaster 6.x-1.0 or higher
- Ubercart 6.x-2.4 or higher
- Patience
Installation
- Download the module and its dependencies into sites/all/modules on your hostmaster platform (
drush @hostmaster dl uc_hosting ubercart
should work) - Activate the clients feature for hostmaster (
admin/hosting/features
) - Enable the module uc_hosting_products. This should enable all dependencies (
drush @hostmaster en uc_hosting_products
)
Creating your first site product
N.B. A video introduction covering configuration is available from the DrupalCon London session Aegir-based Business Models (starting at 26:30).
- Create a product. Make sure it is not shippable. (
node/add/product
) - Edit the product. Go to the features tab.
- Choose the "Create a site and adjust quotas accordingly" feature. It should use the sku of your product automatically.
- For single-site products, make sure, on the product feature form, to check off "Force clients to create their site on purchase" if you want to be sure people create their site.
- Now all you need is a drupal platform that any user can create sites on and you are ready to go. (
node/add/platform
)
Creating more complex offerings using product kits
The method we recommend for creating complex offerings, including multiple site quotas and access to restricted platforms, is via Ubercart product kits.
- Enable uc_product_kit.
- Create one or more uc_hosting products.
- Create a product kit with these products in them in the appropriate quantities.
Support
You can report bugs, request support and propose patches via our issue queue on Drupal.org.
Development
If you would like to use uc_hosting to develop your own Aegir dependant product features, be sure to read the relevant documentation.
Check out the roadmap.
Other Resources
- Aegir community site
- Ubercart Documentation
- #aegir IRC channel on freenode
Provision CiviCRM
provision_civicrm helps to manage the installation and upgrades of CiviCRM in Drupal sites managed with Aegir.
CiviCRM is not a typical Drupal module. It is a third-party system that integrates with Drupal, WordPress and Joomla for its front-end and permission system. CiviCRM has its own installer, upgrade system and cron. The "provision_civicrm" module helps to manage any CiviCRM+Drupal site as any other Drupal site managed in Aegir.
Requirements
- Aegir 2.x or 3.x
- hosting_civicrm must be installed in the hostmaster front-end (~/hostmaster-x-y/sites/example.org/modules).
Installation
Download the provision_civicrm module as the "aegir" user:
drush dl provision_civicrm-6.x-2.x
This should install the module in ~/.drush/provision_civicrm.
NB: provision_civicrm 6.x-2.x is a Drush module, and works with both Aegir 2 and Aegir 3. However, for Aegir 3, you must use the 7.x-3.x branch of the hosting_civicrm module.
Creating a CiviCRM platform
provision_civicrm assumes that if you have a CiviCRM installation in your sites/all/modules/civicrm, then this is a CiviCRM platform and CiviCRM should be installed automatically.
(A cleaner way, in theory, would be to have an install profile and let the user configure it through the front-end, patches welcome! -- however, you do not want any site to simply enable CiviCRM from the Drupal admin/build/modules, so having dedicated platforms for CiviCRM makes sense in any case.)
A sample drush makefile is provided with provision_civicrm.
Assuming you are running as the user 'aegir', whose $HOME is /var/aegir/, save this in a file such as ~/makefiles/drupal-6.22-civicrm-3.3.4.make
Then create the new platform:
mkdir ~/platforms/drupal-6.22-civicrm-3.3.4
cd ~/platforms/drupal-6.22-civicrm-3.3.4
drush make ~/makefiles/drupal-6.22-civicrm-3.3.4.make
You can now add your platform in the Aegir front-end and then create new sites.
Create a SUDO user for server administration
Sometimes you will need to administer your server beyond what you can do as the Aegir user.
Examples include installing additional software or editing PHP configuration.
It's not recommended to use the root
user for administering things as it can be easy to make mistakes that could break your system.
Instead of running as root, it is recommended to create a user for yourself that has sudo privileges.
Create a user with SUDO privileges
In ubuntu/debian systems, creating a user and giving them the ability to run commands with sudo
is easy with the adduser
command.
Think of a username and password for yourself, then run the following commands as root:
root@aegir:~# adduser bob Adding user `bob' ... Adding new group `bob' (1001) ... Adding new user `bob' (1001) with group `bob' ... Creating home directory `/home/bob' ... Copying files from `/etc/skel' ... Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully Changing the user information for bob Enter the new value, or press ENTER for the default Full Name []: Bob Room Number []: Work Phone []: Home Phone []: Other []: Is the information correct? [Y/n] y root@aegir:~#
Then add them to the "sudo" user group:
root@aegir:~# adduser bob sudo Adding user `bob' to group `sudo' ... Adding user bob to group sudo Done. root@aegir:~#
Then you can switch to the "bob" user using sudo su
, or login via ssh with your password.
root@aegir:~# sudo su - bob bob@aegir:~$
Experimental: Aegir 2 on Ubuntu 12.04 with apache, php-fpm
These notes describe a possible way of installing Apache with fastcgi using php5-fpm. The below configuration works but is experimental.
On a Ubuntu 12.04 box:
Install Aegir2 from Debian packages
Follow http://community.aegirproject.org/installing/debian. Below is a shortened form.
Add the Aegir package "Software Source" repository:
echo "deb http://debian.aegirproject.org stable main" | sudo tee -a /etc/apt/sources.list.d/aegir-stable.list
wget -q http://debian.aegirproject.org/key.asc -O- | sudo apt-key add -
Install aegir2:
sudo apt-get update
sudo apt-get install mysql-server
sudo mysql_secure_installation
sudo apt-get install aegir2
If the installation fails, it might be that your FQDN is not set or your postfix is not configured.
Verify your aegir site is working
Navigate your browser to the FQDN of your aegir server. You need to see the "Welcome to Aegir" page.
Modify the configuration to use fastcgi / php5-fpm
Edit /etc/apt/sources.list
and enable the multiverse repository (it's needed for the libapache2-mod-fastcgi package).
Then install php5-fpm and make apache use it:
sudo apt-get update
sudo apt-get install php5-fpm libapache2-mod-fastcgi
sudo apt-get --purge remove libapache2-mod-php5
Modify /etc/php5/fpm/pool.d/www.conf
Make it listen on socket, instead of IP:
;listen = 127.0.0.1:9000
listen = /var/run/php-fpm.sock
Create /etc/apache2/conf.d/php-fpm.conf
with this content:
ScriptAlias /php5.fastcgi /var/www/fastcgi
FastCGIExternalServer /var/www/fastcgi -socket /run/php-fpm.sock
AddHandler php-fastcgi .php
Action php-fastcgi /php5.fastcgi virtual
Options FollowSymLinks ExecCGI
<Directory /var/www/fastcgi>
Options ExecCGI FollowSymLinks
SetHandler fastcgi-script
Order allow,deny
Allow from all
</Directory>
Enable apache modules:
sudo a2enmod actions alias fastcgi
Restart php5-fpm and apache to load the new configs:
sudo service php5-fpm restart
sudo service apache2 restart
Verify that you can still reach your Aegir site
Navigate your browser to the FQDN of your aegir server. This time, you are using php5-fpm. Verify e.g. via phpinfo().
--
Page author: Marji Cermak marji@morpht.com, http://morpht.com
Using a proxy cache to accelerate platform builds
While building (and re-building) platforms using makefiles and the Aegir front-end (node/add/platform) is great, it has a number of minor annoyances. They're slow, and consume lots of bandwidth, but worse still, even a small hiccup in drupal.org (or other) infrastructure can break a platform build, which requires starting from scratch each time.
When you maintain platforms with upwards of 300 modules, themes and libraries, such as with kPlatforms, and you start to deploy these to multiple clients' Aegir servers, those minor annoyances can become significant challenges.
There is good documentation on setting this up for use with Drush Make already, such as this blog post. However, there are some gotchas when doing so in Aegir. Most of the suggested solutions involve something like setting environment variables in ~/.bashrc, which works fine if you're running 'drush make' yourself in a login shell. However, since the Aegir is running in a non-interactive shell, there isn't a opportunity to set environment variables.
Thankfully, Drush Make supports curl, and opts for it automatically if it's installed. Careful reading of curl's manpage indicates that:
[curl] always [...] checks for a default config file and uses it if found.
So, in the end, all that's required is a /var/aegir/.curlrc file containing:
proxy = "http://<proxy_host>[:port]"
Thus a basic configuration on Debian would involve installing a proxy, such as Squid:
# apt-get install squid3
Then add an acl in /etc/squid3/squid.conf allowing access from your local network, e.g.:
acl localnet src 192.168.0.0/16
http_access allow localnet
Restart Squid and you're good to go:
# /etc/init.d/squid3 restart
Testing that Aegir is actually using your new proxy cache, and that Squid is actually caching the makefile projects involves taking a quick look at the logs of the Squid server:
# tail -f /var/log/squid3
Upgrade Guide
1. Introduction
Upgrading Aegir is designed to be as easy as it possibly can be, and regularly improves over time as the developers attempt to streamline the process for users.
Nonetheless, the upgrade process can seem a little daunting to users. This is mainly because some expectation arises that because Aegir is built on Drupal, and is made for managing Drupal sites, it seems reasonable to expect the upgrade process to be as straightforward as, say, upgrading a regular Drupal site.
Aegir is a powerful system that goes beyond a normal Drupal application by being split into two parts: a frontend (the browser-based control panel) and the backend (bits on the system in /var/aegir, such as Drush and Drush extensions), along with a system user on your server that runs command-line scripts and restarts services, and cronjobs.
Typically what this means is that when it comes time to upgrade Aegir, you not only have to upgrade the frontend site, but also update the components that reside in the 'backend'.
But don't panic! We have instructions and even a script to run, to take care of almost all of it for you, eliminating as much as possible the chance for human error.
2. Upgrade options
There are three options you can choose to upgrade your Aegir install when a new release comes up:
3. New release of Drupal core?
Just as you would use the Migrate task in Aegir to upgrade one of your sites to a new copy of Drupal core, you can follow the UPGRADE.txt to upgrade your actual Aegir frontend system to a new copy of Drupal core too. The upgrade process (including the script) will always fetch the latest available copy of Drupal core to place the frontend on.
Currently you cannot upgrade the Aegir site itself from the frontend with a Migrate task as you do for your managed sites. This may change in the future, but has not been developed yet.
Manual Upgrade
This section outlines the requirements for doing a manual upgrade of Aegir to a new release. Before going ahead with this, you probably want to read the upgrade path and version-specific notes.
1. Conventions & Tips
All instructions and in general all commands must be run as aegir user, so all permissions are always set correctly.
To become aegir user you can issue this command:
su -s /bin/sh aegir
(Note you must run this as root or prefix with sudo).
Note that /bin/sh is an example. You may wish to instead use the shell of your choice, i.e /bin/bash
A standard umask of 022 is assumed. This is the default on most systems.
We assume that drush is in your path, if it is installed in /var/aegir/drush/drush.php
, replace all occurences of drush
by /var/aegir/drush/drush.php
in the following steps, or add the following alias:
alias drush=/var/aegir/drush/drush.php
2. Upgrading Overview
2.1. Upgrading drush
As part of your Aegir upgrade, you need to upgrade Drush to the latest stable version compatible with the target Aegir release. AEgir 1.x only works with version 4.5. Versions 5 - 7 have been known to work with AEgir 2.x, however the Drush developers recommend version 6 as the most stable.
2.2. Upgrading the backend
After drush is updated, you must then proceed to upgrade the backend. The reason for upgrading the backend first before the frontend, is that the frontend upgrade process is instigated by the backend using Drush Make. Thus you need to be running the new backend first in order to successfully produce a new frontend.
In general, we try to keep the backend and the frontend compatible with each other during release cycles. That is: provision 1.8 and hosting 1.9 will always be able to talk to each other, but hosting 2.0 will have trouble talking to the 1.9 backend, so you need to upgrade the backend first when you do a major upgrade (for example, 1.9 -> 2.0).
Bottomline: first you upgrade the backend, then the frontend.
2.3. Upgrading the frontend
Once your backend is upgraded, you can upgrade your frontend. Think of this as the backend fetching a new copy of Drupal core and the Hostmaster frontend application onto your server, and then moving the Aegir site's settings.php and other bits and pieces over to the new codebase.
2.4. What hostmaster-migrate does
The command will make sure the target directory is a valid Aegir install. If the directory does not exist, provision will use drush_make to fetch and assemble the correct version of the front end for the specific release of the backend you are running.
hostmaster-migrate
will also completely replace the crontab entry for the aegir user.
The command above will fetch the latest stable Drupal release, so it can simply be run again when a new security release of Drupal is made available.
3. Upgrading to AEgir 1.x
3.1. Upgrading drush and drush make
As part of your Aegir upgrade, you need to upgrade Drush to the latest stable version compatible with the target Aegir release. Aegir 1.x does not support Drush 5, although 2.x does.
If you are using the Drush Debian packages, the package should be upgraded through:
apt-get install drush
Otherwise, since drush 4.x is capable of upgrading itself, you can just run:
drush dl drush-7.x-4.5
If you are running a version before 4.x, use the following set of commands instead:
pear channel-discover pear.drush.org
pear install drush/drush-4.5.0
Provision 0.4 has added a new dependency on drush_make, which will also need to be installed to upgrade the front end if you are upgrading from a pre-0.4 release.
If you are upgrading from an earlier 0.4 release, replace your copy of drush_make with the latest recommended release.
drush dl drush_make-6.x --destination="$HOME/.drush"
3.2. Upgrading the 1.x backend
Upgrading the backend is as simple as replacing your copy of Drush, Provision, and if necessary, Drush Make, with the new versions if available. Keep a copy of the old Provision and Drush in case something goes wrong in the frontend.
drush dl provision-6.x --destination=$HOME/.drush
3.3. Upgrading the 1.x frontend
Before upgrading, we set a few key variables to make the process easier. You must replace the arguments those variables to match your configuration. This means replacing $AEGIR_DOMAIN
by the URL of your Aegir install, $AEGIR_VERSION
with the version number of the Aegir you are trying to install, and also $OLD_AEGIR_DIR
with the path to the previous directory where Aegir was installed.
The directory passed to hostmaster-migrate
must be an absolute path to where you want the new release to be stored.
OLD_AEGIR_DIR=/var/aegir/hostmaster-6.x-1.8
AEGIR_VERSION=6.x-1.11
AEGIR_DOMAIN=aegir.example.com
To upgrade your frontend, run the following commands, replacing the variables as described below:
cd $OLD_AEGIR_DIR
drush hostmaster-migrate $AEGIR_DOMAIN $HOME/hostmaster-$AEGIR_VERSION
3.4. Special configurations and troubleshooting
If the upgrade fails, try running it again with the --debug
flag:
cd $OLD_AEGIR_DIR
drush hostmaster-migrate aegir.example.com $HOME/hostmaster-$AEGIR_VERSION --debug
If you have customized your Aegir installation and are maintaining your own makefile, you can use the --makefile
flag so the platform is created with another makefile than the default. Be warned that this may create problems if the makefile doesn't include the right Aegir modules.
4. Upgrading to AEgir 2.x
4.1. Upgrading drush
Install Drush 6.x per the directions given at the Drush repository. The maintainers have deprecated the PEAR installation procedure and are encouraging everyone to install Drush via the Composer dependency management tool.
You should also remove your copy of drush_make
from $HOME/.drush
.
4.2. Upgrading the 2.x backend
Upgrading the backend is as simple as replacing your copy of Drush & Provision with the new versions if available. Keep a copy of the old Provision and Drush in case something goes wrong in the frontend.
drush dl --destination=/var/aegir/.drush provision-6.x-2.0
Drush 5+ has a commandfile cache which you need to clear before installing Aegir:
drush cache-clear drush
4.3. Upgrading the 2.x frontend
Before upgrading, we set a few key variables to make the process easier. You must replace the arguments those variables to match your configuration. This means replacing $AEGIR_DOMAIN
by the URL of your Aegir install, $AEGIR_VERSION
with the version number of the Aegir you are trying to install, and also $OLD_AEGIR_DIR
with the path to the previous directory where Aegir was installed.
The directory passed to hostmaster-migrate
must be an absolute path to where you want the new release to be stored.
OLD_AEGIR_DIR=/var/aegir/hostmaster-6.x-1.11
AEGIR_VERSION=6.x-2.0
AEGIR_DOMAIN=aegir.example.com
To upgrade your frontend, run the following commands, replacing the variables as described below:
cd $OLD_AEGIR_DIR
drush hostmaster-migrate $AEGIR_DOMAIN $HOME/hostmaster-$AEGIR_VERSION
4.4. Special configurations and troubleshooting
Read the upgrade path notes again, you will have missed something.
If the upgrade fails, try running it again with the --debug
flag:
cd $OLD_AEGIR_DIR
drush hostmaster-migrate aegir.example.com $HOME/hostmaster-$AEGIR_VERSION --debug
If you have customized your Aegir installation and are maintaining your own makefile, you can use the --makefile
flag so the platform is created with another makefile than the default. Be warned that this may create problems if the makefile doesn't include the right Aegir modules.
If the update fails due to a bug in module Hosting, it will ask you to run "drush hostmaster-resume help". That command does not exist. Try this one instead.
drush --debug @hostmaster hostmaster-resume /var/aegir/hostmaster-6.x-1.11 /var/aegir/hostmaster-6.x-2.0
Automatic upgrades with Debian packages
1. Regular update process
Make sure you have the Aegir repository in your sources.list, as per the installation docs.
If you are upgrading from a Debian installation whereby you used the koumbit.org repositories, you should change this entry in your sources.list to be debian.aegirproject.org as per the instructions linked to above, and run apt-get update to refresh.
If you want to upgrade all packages in one shot, use:
apt-get update
apt-get install aegir
Note that you can upgrade Aegir in steps. Upgrading your backend to the Debian package should be as simple as running:
apt-get install aegir-provision
Upgrading the frontend to the Debian package works identically:
apt-get install aegir-hostmaster
As during install, you can use the DEBUG variable to run Drush in debugging mode:
env DEBUG=yes apt-get install aegir-hostmaster
Note: the DEBUG flag is deprecated. In 1.x releases, it is accepted, but in 2.x it won't. You should use DPKG_DEBUG instead, it is usable starting from 1.1-4.
2.x note: to upgrade to 2.x, use: apt-get install aegir2
.
Note on contrib modules: some Aegir-specific contrib modules may have been installed on your Aegir deployment. This may fail if you are doing a major upgrade. In this case, you will probably want to disable those contrib modules. Look for directories in ~aegir/.drush
or /usr/share/drush/commands
or other components of the drush commandfiles search path, and move them out of the way. After the upgrade, download the new versions of the modules that are compatible with the new Aegir release.
2. Upgrading from non-Debian installs
The Debian package supports migrating from existing installs. You will need to move /var/aegir/.drush/provision out of the way before going ahead:
tar cfz /var/aegir/backups/provision.tgz /var/aegir/.drush/provision
rm -rf /var/aegir/.drush/provision
You'll also need to go through steps 1 through 3 of Automatic install on Debian.
Then just install the package as if you were installing from scratch.
3. Custom distributions
If you have your own makefile, you can go ahead with the above process, but change the makefile to the one you want:
echo debconf aegir/makefile string /var/aegir/makefiles/aegir/aegir-koumbit.make | debconf-set-selections
apt-get install aegir
Usually no questions are asked when upgrading - this allows you to specify the makefile path for your custom distribution of Aegir, even if you're upgrading. It's also how you can switch distributions.
An example aegir-koumbit.make file could look like:
core = 6.x
api = 2
includes[aegir] = "/usr/share/drush/commands/provision/aegir.make"
projects[] = module_filter
Note that for this to work, you will need the patch from this issue, allowing drush_make to reference absolute system paths.
3.1. Keeping custom distributions up to date
This section describes how to upgrade the code for custom distributions in between debian package upgrades. Here, we assume that Aegir is already installed through the debian package.
- Create a new Aegir makefile, or update the custom Aegir makefile. (You can use the makefile in drupal.org/project/provision as a starting point.)
- Add modules/features/themes/libraries/whatever to the makefile. Ex:
projects[] = token
projects[] = views - Follow the manual upgrade path:
- drush @hostmaster hostmaster-migrate URL /path/to/platform --makefile='/path/to/makefile'
- Hope it's all green (no errors). If errors, fix makefile.
- A new platform should exist, the new site should be in there, verifying. If the task is not running, check that hosting-queue-runner is running. If it's not, restart it using '/etc/init.d/hosting-queue-runner force-reload'. (Since hostmaster changed location, it is normal for hosting-queue-runner to crash at this point.)
- When the debian package is upgraded, it should create a new hostmaster platform automatically. (You can test this with dpkg --configure.)
4. Rolling back the upgrade
If something went wrong with the upgrade, you can rollback by deleting the hostmaster site and redeploying on the older platform:
drush @hostmaster provision-delete
vi ~/.drush/hostmaster.alias.drushrc.php
...And edit the alias to make the following changes:
1. change the platform to point to the older platform alias
2. change the platform root path
3. change the site path
During the upgrade, a backup was done and you need to find which backup file it was in the backups directory. Use this backup to redeploy the older platform:
drush @hostmaster provision-deploy ~/backups/hostmaster.date.tgz
5. Recovering a Failed Upgrade
When using the Regular Update Process and you encounter an error it leaves the debian package in a broken state.
First you need to figure out why it failed. For example is there a permission problem?
Once you determined the issue correct it and clean up the environment. It could be necessary to remove the new hostmaster platform. It is up to you to determine what state aegir is in.
Next you need to fix the broken package by running the fix-broken parameter:
apt-get install -f
This tells the debian package system to start through the process again.
Automatic upgrades with upgrade.sh script
This page describes the upgrade script in the Provision repository that tries to automate much of the upgrade process.
NOTE: At this time neither the Aegir 1.1 nor the Aegir 2.0 upgrade scripts are working for all setups. We recommend you use the Manual upgrades instead.
It is imperative that you read the version-specific upgrade notes before attempting to run the upgrade.sh script, as the script will assume you have your system set up appropriately to handle the upgrade process.
You can download the upgrade.sh script for Aegir 2.4 with the following command:
wget -O upgrade.sh 'http://drupalcode.org/project/provision.git/blob_plain/refs/tags/6.x-2.4:/upgrade.sh.txt'
or for AEgir 3.0 with the following command:
wget -O upgrade.sh 'http://drupalcode.org/project/provision.git/blob_plain/refs/tags/7.x-3.0:/upgrade.sh.txt'
Make sure you download it to somewhere that the aegir user can access in order to execute it.
You may need to edit the script to set any variables that are different from the defaults, for example to upgrade to a different Aegir version.
To do the upgrade, just run the script:
su -s /bin/sh aegir -c "sh upgrade.sh"
Version-specific & upgrade path information
- 1. General upgrade path recommendations
- 2. Changes in the 2.0 release
- 2.1. hosting-queue-runner renamed hosting-queued
- 2.2. queue daemon enabled by default
- 2.3. hosting_platform_pathauto module bundled with Aegir
- 2.4. hosting-task --force
- 2.5. Deprecated functions removed
- 2.6. New Provision class naming convention
- 2.7. Client node type changes
- 2.8. SSL and IP allocation refactoring changes
- 2.9. Debian package name changed
- 2.10. Platform drushrc.php moved
- 2.11. Class auto-loading
- 2.12. Other database changes
- 3. Changes in the 1.0 release
1. General upgrade path recommendations
In general, you shouldn't skip major releases. That is, if you are running 0.3, you should upgrade to 1.0 or 1.1, not 2.0. Upgrades within a stable branch (1.0 to 1.1, 2.0 to 2.1) are fully supported, and you can skip minor versions (1.0 to 1.2 should work fine), but you should always upgrade to the latest stable release if you are coming from a different release branch.
The rest of this page documents what version-specific changes happened that you need to make manual modifications to. If you are running the Debian packages, you probably don't need to configure anything here.
The upgrade process should make this clear, but keep in mind that you need to upgrade the backend (drush, provision) before the frontend (drupal, hostmaster profile, etc).
Provision/Drush contrib modules might require additional attention. As there is no version checking, the safest method is probably to move the module out of your .drush folder and later add the version compatible with the new Aegir version.
Hosting/Drupal contrib modules should also be handled carefully. If these were installed into the hostmaster site's directory, then they should be moved automatically to the new hostmaster platform during the upgrade. On the other hand, if they were installed on the platform, you'll need to provide the hostmaster-migrate or Debian package with a custom makefile. The default makefile in Provision (aegir.make) can be used as a model for this.
2. Changes in the 2.0 release
2.1. hosting-queue-runner renamed hosting-queued
The popular module hosting-queue-runner
has been merged in the 2.0 release. In the process, it has been renamed to the more appropriate hosting-queued
. The init script is also now installed directly with the Debian package, and started on install or upgrade.
You will, however, need to manually enable to module for the daemon to work (still):
drush @hostmaster en hosting_queued
Also, if you are migrating from an install using hosting-queue-runner
, you will need to remove the old module and startup script. Enabling hosting-queued
will already have disabled and uninstalled the module from Drupal, but you need to cleanup your filesystem:
rm /etc/init.d/hosting-queue-runner
2.2. queue daemon enabled by default
The above queue daemon is now enabled by default by the Debian package, and is properly restarted on upgrades. It can be disabled after install if necessary using the following comamnds:
service hosting-queued stop
su aegir -c "drush @hostmaster pm-disable hosting_queued"
2.3. hosting_platform_pathauto module bundled with Aegir
The popular module hosting_platform_pathauto
has been bundled in the 2.0 release.
If you installed it module manually then now is probably the time to remove your copy.
Aegir 2.0-alpha1 comes bundled with hosting_platform_pathauto-2.0-beta1.
2.4. hosting-task --force
You used to be able to run hosting-task <nid>
to re-run a task by its node ID. Because hosting-task now checks if the task is really queued, this can't be used to retry existing tasks directly, unless you use the --force
argument. So this:
drush @hostmaster hosting-task 1234
becomes:
drush @hostmaster hosting-task --force 1234
2.5. Deprecated functions removed
If you are using any of those functions in your custom code, lookup in the 1.x documentation to see the replacement functions, as they have been removed from 2.x:
- hosting_alias_count_sites() (same as !hosting_alias_allow_domain())
- hosting_site_exists() (same as !hosting_alias_allow_domain())
- hosting_get_client_by_email() (use hosting_get_client() instead)
- hosting_import_client() dropped the second argument (organization)
- hosting_task_list_embedded() (use hosting_task_list() or hosting_task_list_table())
2.6. New Provision class naming convention
Provision classes used to start with provisionConfig_ or provisionService_, but these are now Provision_Config_ and Provision_Service_ respectively.
2.7. Client node type changes
- the email field was removed from the client, now you need a user associated with the client. this was already the case in 1.x but the existing field wasn't touched.
- the client_email field was removed from the site object
2.8. SSL and IP allocation refactoring changes
The SSL IP allocation refactoring has changed a good few things in the API and the way SSL certificates are managed.
2.8.1. Backend
The following functions are gone from the backend:
- provisionService_http_ssl::assign_certificate_ip
- Provision_Service_http_ssl::get_ip_certificate .
- Provision_Service_http_ssl::free_certificate_ip
- Provision_Service_http_ssl::clear_certs()
Certificates are not tracked from the backend anymore, other than to make sure that we delete a certificate when it's not used anymore. So we still use *.receipt files, but just to mark a site to SSL certificate association, not the IP allocation. The SSL directory in ~/config/server_foo/ssl.d/uri/
is removed when no site use that directory anymore.
The backend now expects the frontend to send an ip_address
field and stopped sending a site_ip_addresses
to the frontend.
2.8.2. Frontend
The hosting_ip_addresses
table changes. A unique key has been added and it is now only for servers. It is used by the new hosting_ssl_cert_ips table to map those server IP addresses to certificates.
A new field for the IP (ip_address) is sent from the frontend during site install/verify tasks. The field is not present in non-ssl sites. It is the non-resolved IP address of the server assigned to the SSL certificate and must be used by the backend to write the vhost.
SSL certificates are removed from the frontend when sites using it are deleted.
2.9. Debian package name changed
To allow both packages to coexist in the same Debian archive, the name of the packages of the 2.x series have changed.
aegir -> aegir2
aegir-provision -> aegir2-provision
aegir-hostmaster -> aegir2-hostmaster
aegir-cluster-slave -> aegir2-cluster-slave
2.10. Platform drushrc.php moved
From Drush 5 documentation:
Drush 5 no longer looks for aliases, configs or command files in the Drupal
root folder, so if you previously used drushrc.php files in the Drupal root
you will need to move the file to sites/all/drush/drushrc.php.
Since Aegir 2 requires Drush 5 (or later), Provision now follows this standard, and places a platform's drushrc.php in sites/all/drush/drushrc.php.
2.11. Class auto-loading
Provision has been refactored to use class autoloading. This simplifies use of classes, as it obviates the need to explicitly include the files in which the class and it's parent classes are defined. In addition, it standardizes the placement of class definitions within a hierarchical file-system structure and will make name-spacing simpler if/when we begin to adopt Symfony components.
More info in the manual under Extending Aegir - Class auto-loading
This may also affect certain contrib extensions that used to have to reside in the provision directory itself. These extensions should probably be removed prior to the upgrade, and re-installed afterwards with an Aegir2-compatible version.
2.12. Other database changes
The release_id
column was dropped. This column wasn't used anywhere in the code and was present only in some installs, see issue #1882708 for details.
3. Changes in the 1.0 release
3.1. Git dependency
If you intend on upgrading your system to the bleeding edge version of the code from our git repositories, you will need the git program installed. On Debian, this means:
apt-get install git-core
3.2. DNS Configuration
Aegir requires that the hostname returned by the hostname
and uname -n
shell commands, resolves to the IP address for this server.
Shell commands as root:
AEGIR_HOST=`uname -n`
resolveip $AEGIR_HOST
If the command returns your IP address, you are all set. If it returns
an error you will need to edit your /etc/hosts
file.
(Mac OSX notes: if you are running on OS X you will not likely have a resolveip binary. Just substitute
ping $AEGIR_HOST
).
First line of this file looks like:
127.0.0.1 localhost
Simply add all domains you want to this line. e.g:
127.0.0.1 localhost $AEGIR_HOST $AEGIR_DOMAIN other1 other2
To be clear you cannot put $AEGIR_HOST $AEGIR_DOMAIN in your hosts file. Put in whatever uname -n returns for $AEGIR_HOST and what ever domain you want to use f or $AEGIR_DOMAIN (real domain or faked in hosts file). In OS X uname -n will return whatever you put in System Preferences -> Sharing -> Computer Name. I added that value (iMac 27 for me) to /etc/hosts and my faked aegir domain as follows:
127.0.0.1 localhost iMac27 aegir.local
If you only intend to use Aegir on a single server, it is acceptable for the resolved IP address to be the '127.0.0.1' loopback address.
If you intend to manage multiple servers using Aegir, you will need to make sure that the IP address is the public IP of this server.
Finally, set an $AEGIR_IP environment variable for use in the Database configuration step below.
Shell commands as root::
AEGIR_IP=resolveip $AEGIR_HOST | awk {'print $6'}
(If running OS X or are otherwise missing resolveip and you just want to use Aegir on your computer use you can do this: AEGIR_IP=127.0.0.1
3.3. Unzip dependency
As of the 0.4-alpha3 release, 'unzip' is a required dependency on your server in order to successfully extract the jquery.ui library that is part of some UI improvements. On Debian, this means:
Shell commands::
apt-get install unzip
3.4. Database configuration
To make sure that the Aegir backend, and all the possible web servers can reach your database server, you need to configure mysql to listen on all the public IP addresses available to it.
/etc/mysql/my.cnf configuration line to comment out:: bind-address = 127.0.0.1
Now you need to restart mysql, to clear any caches.
Shell command as root:: /etc/init.d/mysql restart
Because you have already installed Aegir when it was using the generic grants, you will need to create new grants using the public IP address and hostname of this server.
Shell command :: mysql -uroot -p mysql
You need to generate the following grants using the hostname returned by the
uname -n
command, and the IP address that the resolveip
command returns
for that hostname.
You need to re-use the pasword you had for the account before.
Shell commands::
mysql -u root -p -e "GRANT ALL ON *.* to 'aegir_root'@'$AEGIR_HOST' IDENTIFIED BY 'xxxx' WITH GRANT OPTION;"
mysql -u root -p -e "GRANT ALL ON *.* to 'aegir_root'@'$AEGIR_IP' IDENTIFIED BY 'xxxx' WITH GRANT OPTION;"
3.5. Apache configuration
This release introduces multi-server support and required reorganizing the Apache configuration files in ~aegir/config. Instead of having all files in config/vhost.d, they are now split between vhost.d, platform.d and a single apache.conf. The vhost.d directory is for virtual hosts, platform.d is for platform-specific configuration and apache.conf is the server-wide configuration file.
You will need to change the line you added to either the httpd.conf file or /etc/apache2/conf.d/aegir file during installation.
Open your httpd.conf file and modify::
Include /var/aegir/config/vhost.d
To read ::
Include /var/aegir/config/apache.conf
If you are upgrading from 0.4 releases between alpha8 and (including) alpha14, you will need to rename your conf.d directory to post.d in Apache and pre.d in Nginx. Example, in Apache::
mv /var/aegir/config/server_master/apache/{conf.d,post.d}
Now log into Aegir, and verify the hostmaster platform. This will generate the correct apache.conf file and restart Apache.
Uninstalling Aegir
There is no formal method for uninstalling Aegir, but there is also no real mystery either since the system tries to keep itself together in one location, typically /var/aegir.
Below are the steps to completely remove all traces of Aegir from your server. We also include to leave Aegir intact, but destroy its data, attempting to reset to a almost post-install setup.
Ideally, you would delete all sites and all platforms from the system first. That would take care of deleting all databases and users. But this can take some time and is annoying, so we provide instructions on how to destroy everything.
WARNING
Obviously if you have any sites and platforms currently managed by Aegir, you would want to move them out of the /var/aegir area before you delete it! Ensure you have set up your sites and platforms elsewhere, with Apache vhost files etc in their typical locations, or at least out of harm's way before attempting this.
WARNING: Performing these steps will remove Aegir from your server and may result in loss of data. Use with caution.
Run these commands as a privileged user (such as root). We assume your Aegir installation resides in /var/aegir.
Backup
You probably want to backup everything before trashing it.
Backup all you need to keep from the /var/aegir/ directory.
Backup your database server, we'll be also destroying that.
Uninstall script
Aegir 2.x comes with a hostmaster-uninstall
script that allows you to automate good chunks of the steps below.
Destroying the data
Aegir manages three types of data:
- configuration files - yes, we consider those as data
- site and platform files
- site databases
Trashing configuration files
This step is optional if you are going to remove everything anyways.
rm /var/aegir/.drush/*alias.drushrc.php rm -r /var/aegir/config/*
Trashing site and platform files
At this point, you could still probably recover by verifying the platforms, which would recreate config files. At this point we start really destroying data.
rm -r /var/aegir/platforms/* rm -r /var/aegir/hostmaster-*
Drop the databases and db users
There are no generic instructions for this, since every system differs. Essentially you can perform this within a MySQL shell
mysql -u root -p
Use the DROP DATABASE $databasename;
syntax to drop databases. and DROP USER $user;
to delete users, example:
DROP DATABASE aegirexamplecom;
DROP USER aegirexamplecom@localhost;
This removes the Aegir user, assuming you have not created any site. If you did, you may want to cleanup those sites if you are going to delete them anyways. To see a list of GRANTs that Aegir has made for your database users, you can use a command like the following:
USE mysql;
SELECT User,Host FROM user WHERE Host='localhost' AND User <> 'root' AND User <> 'debian-sys-maint';
SELECT Host,Db,User FROM db WHERE User <> '';
In Debian, the following will remove all non-system users (which may include non-aegir users!!!).
WARNING: DO NOT RUN THIS BEFORE CHECKING WHAT WILL BE DELETE BY RUNNING THE SELECT ABOVE!! THIS WILL DELETE ALL USERS FROM YOUR MYSQL DATABASE!!!
DELETE FROM db WHERE User <> '';
DELETE FROM user WHERE Host='localhost' AND User <> 'root' AND User <> 'debian-sys-maint';
FLUSH PRIVILEGES;
Then you will need to DROP DATABASE
for every database.
Consult the GRANT and DROP documentation from MySQL for more information.
Remove the aegir user's crontab
crontab -r -u aegir
Removing everything
At this point, we have removed all data that Aegir manages and we would be ready to reinstall the frontend. If we want to remove all traces, however, there's more work to do.
Remove /var/aegir
rm -rf /var/aegir
Delete the aegir user
userdel aegir
This will also remove the user from the www-data group.
Remove the user from sudoers
visudo
Remove the line that looks something like
aegir ALL=NOPASSWD: /usr/sbin/apache2ctl
Save and exit the file.
Delete the symlink/include from Apache
Depending on your installation and OS this may vary.
Generally:
rm /etc/apache2/conf.d/aegir.conf
If your Include statements were contained in a global system httpd.conf
file or similar, you will need to remove these lines manually.
Restart Apache when you have completed this.
Extracting a Drupal site from Aegir
You may occasionally want to take a Drupal site hosted on a server managed with Aegir and put it somewhere else. This is pretty easy, but not quite as simple as moving a normal Drupal site not managed by Aegir from one machine to another. In brief, you have to copy the site, delete the drushrc.php file, and replace settings.php with a copy of the default configuration file. More specifically:
- download the same copy of drupal core to the new server (and re-apply core patches, if any)
- copy sites/all/ to the new server
- copy the Aegir backup of your site to the new server and unzip in sites/yoursite.com/
- create a database on the new server & import the database backup
- copy sites/community.aegirproject.org/default.settings.php to sites/yoursite.com/settings.php
- D7: copy custom site aliases in sites/sites.php (if any) to the new server
- D6: replicate site alias symlinks in sites/ (if any) on the new server
- copy custom configuration in local.settings.php (if any) to settings.php
- edit settings.php to add the database authentication details for the new server
That's it!
Troubleshooting Aegir
Aegir is a complex system that depends on a number of components being properly integrated. This can obviously lead to difficulties on occasion. However, fear not! In all likelihood, any problem you can get yourself into has probably already been solved, and documented in this section. Just one of the benefits of using a popular open source system ;)
How to fix a stuck task
It is possible a task may run longer than expected and you suspect something has gone wrong. It is even possible the task may cause a server to become unresponsive. Restarting the server will not stop the task, although it may make the server become responsive again.
To stop a Task, find the related task node (install: mysite.com) in the content list: /admin/content/node/overview, and delete it. This will stop Aegir from trying to run the task.
In the case of a site install, it may also be necessary to also delete the node for the site itself. The site may have actually installed properly. Verify the affected platform and see if Aegir imports the site. If so, you may need to reset the password to gain access to the new site. If the site doesn't import in the verify, delete any files from the sites folder on the selected platform and either try again and/or research why the task may have failed.
It can also be very helpful to try running tasks from the command line with drush (the entire Aegir web frontend is basically a user-friendly interface to a series of drush commands). You can see which command is being run by looking at a task node in Aegir. For example, if you are having problems verifying a site, you could run:
drush @mysite.com provision-verify --debug
Another option is to run the hosting task itself within the hostmaster context, which will then invoke the drush command on the applicable site. You can find the task node ID in the web frontend (look at the URL generated by Aegir to create the popup box which shows the log). If the task ID was 1234, you would then run this from the command line:
drush @hostmaster hosting-task 1234 --debug
Debugging Aegir
Installation and configuration of Aegir has been greatly simplified with the release of .deb packages. However, for other operating systems, or for those installing manually, making a mistake during the installation or upgrade of Aegir or Drush is easy to do, and these tips identify some of the most common mistakes.
The Aegir User
- Make sure you have an aegir user (who we call the Aegir User) and that the Aegir User is properly configured on your server:
cat /etc/passwd
prints all users. Find the lineaegir:x:1000:1000:,,,:/var/aegir:/bin/bash
.aegir
usually is the Aegir User. Be cautious about using a different username, and do notuse an existing user or a user that will have any other role on the server. The Aegir User should only be used for Aegir activities.- Verify the Aegir install directory is the user's home. Usually, the Aegir directory is
/var/aegir
. Be cautious about installing Aegir in a different directory only because you are likely to get confused and nearly all the documentation assumes you have used/var/aegir
- Make sure that the node that defines your Aegir server (e.g.,
www.example.com/node/2
) identifies theScript User
as your Aegir User. (You can find this node by clicking on the Servers menu tab and then clicking on the server URI shown in the list that appears of Aegir servers.) - All this is detailed in the install profile wizard
- Do not run aegir as root! - It's an unsafe and unsupported configuration that is known to cause problems
Make Sure drush Is Installed Properly
- Make sure that the node that defines your Aegir Server has the correct path to drush.
- See
/var/aegir/drush/readme.txt
for install instructions
- See
- Check your Drush setup:
- Always run drush as your aegir user, and never as "root".
- Make sure that the Provision module is in the
aegir/.drush/provision/
directory. - Make sure that the Hosting module is in
~aegir/hostmaster-0.x/profiles/hostmaster/modules/hosting
(substitute the platformhostmaster-0.x
to match the directory on your system in which you have installed the Drupal code that is running your Aegir site) - Run "drush help" and make sure that "provision" commands appear. "hosting" commands will not appear in the latest (after 2009/05/18) Drush versions, unless Drupal can be fully bootstrapped
Apache
- Make sure the Aegir User can restart Apache without a password and that the corresponding command is defined in your Aegir Server Node
- As your Aegir User, run the command shown in the "Restart command" field in your Aegir Server Node to ensure that your Aegir User can restart Apache
- On Ubuntu, it's
sudo /usr/sbin/apache2ctl graceful
. The command should execute and return without asking for a password when run as the Aegir User
- Make sure Apache can parse its configuration
- On Ubuntu,
sudo apache2ctl configtest
should report "System Ok"
- On Ubuntu,
Other Things
- On a Linux system, check that /etc/sudoers is chmod 440 (i.e., read only for the file owner and group, which is usually root)
- Attempt to run the cron tasks manually from the command line
php /var/aegir/drush/drush.php --root=/var/aegir/hostmaster-0.x --uri=http://youraegirsite.com hosting-tasks --debug
- Or you can retry a specific task form the command line
php /var/aegir/drush/drush.php --root='/var/aegir/hostmaster-0.4-alpha5' --uri='youraegirsite.net' hosting-task <nid of the task> --debug
"strict warning" error
Aegir 2.x returns the following error message
strict warning: Non-static method view::load() should not be called statically in /var/aegir/hostmaster-6.x-2.1/profiles/hostmaster/modules/views/views.module on line 1113.
Cause
It's due to a conflict between Views 3.x and PHP 5.4. For example, your server uses PHP 5.4 or more recent. And you updated Drupal core for Aegir to Drupal 6.34 or more recent and this error message is showing on all Aegir pages.
Solutions
Option 1: Install "disable_messages" module. And disable all messages related to:
strict .*.
Read more here, under "How to hide only specific error messages to specific users" section.
Option 2: Patch Views 3.x. Read more at https://www.drupal.org/node/780156 or https://www.drupal.org/node/893128#comment-4760978
Option 3: Turn off Strict warnings in your "php.ini" file. Read more at https://www.drupal.org/node/1833170#comment-6994142 or https://www.drupal.org/node/465332#comment-6533810
Community coordination guide
This section of the manual aims to document common release practices and coordination within the community.
This includes:
- The release process
- The release workflow
- Roadmap planning (version 2.0 and version 3.0)
- Weekly scrums
Release process
This page is being moved to http://docs.aegirproject.org/en/latest/community/release-process/
This page aims to document our release process. It documents the release cycle, but also the steps required to make a release.
- 1. The release cycle
- 2. Steps for a release
- 2.1. Make sure Jenkins is all green
- 2.2. Verify that drupal-org.make specifies up-to-date versions
- 2.3. Generating the release notes
- 2.4. Running the release.sh script
- 2.5. Test the manual install in Jenkins
- 2.6. Build the Debian packages
- 2.7. Test the upgrade in Jenkins
- 2.8. Creating release nodes on Drupal.org
- 2.9. Build the release in Jenkins again
- 2.10. Publish the Debian packages
- 2.11. Publish the release notes widely
1. The release cycle
In general, each major Aegir release comprises a simultaneous release of all the modules that are part of the project. We generally go through several testing releases (alphas, betas & RCs) before doing the first stable release on a branch.
- First an alpha is released to test new functionalities and to accomplish the goals decided in the Project Roadmap for that major version. Example:
0.4-alpha1
,0.4-alpha15
- When we have covered most of the functionalities outlined in the roadmap, we push out beta releases until no more critical issues show up. This is generally considered a soft feature freeze. Example:
0.4-beta1
- Then we go into full feature freeze and release a first release candidate (RC). Then a stable release branch is created, and the development branch is kept opened for development for the next stable release. This is generally considered a soft API freeze. Release candidates are made as long as critical bugs are found. Example:
0.4-rc1
, freeze announcement - Once the development branch has no known critical bugs, the stable release is announced. From there on only critical fixes (security, critical performance and critical bugfixes) are committed to the stable branch, and stable releases are published (without alpha/beta/RC) directly on the stable branch. The stable branch is in full API freeze. New features are generally committed to the development branch.
See also the tag and branch naming convention.
2. Steps for a release
For the specifics of release naming conventions and the cycle, see the branch naming convention.
2.1. Make sure Jenkins is all green
Look into Jenkins to see if all tasks have been performed without errors since the last commit. If there is an error, fix it before the release.
2.2. Verify that drupal-org.make specifies up-to-date versions
In the hostmaster project we maintain our own drupal-org.make file. Check that e.g. the ctools version specified is not out-dated.
2.3. Generating the release notes
We build complete release notes for every release. Those are made up of a summary of the release, an outline of key changes, of known issues, install and upgrade instructions and a full list of bugfixes and new features.
Using Git Release Notes for Drush
drush rn 7.x-3.0-alpha1 7.x-3.0-alpha2
The developers then proceed to format/edit the list of fixes as well as list other significant information/changes for this release. These notes end up becoming the Release Notes for the release, which are also entered in the debian/changelog
file by the script below.
2.4. Running the release.sh script
Each time we make a new release, we run a script called release.sh
in provision.
This script should only be used by the core dev team when doing an official release. If you are not one of those people, you probably shouldn't be running this.
This script does all the 'hard' work in that it doesn't forget all the very many places to edit version numbers etc of relevant documentation and other scripts. This includes install.sh.txt and upgrade.sh.txt.
note from just after 7.x-3.0-beta1: It's probably a good idea to disable the Jenkins jobs named 'D_aegir-debian*'. This could prevent later troubles in the 'Publish the Debian packages' section.
Paraphrasing from the script itself:
The following operations will be done:
0. prompt you for a debian/changelog entry
1. change the makefile to download tarball
2. change the upgrade.sh.txt version
3. display the resulting diff
4. commit those changes to git
5. lay down the tag
6. revert the commit
7. (optionally) push those changes
The operation can be aborted before step 7. Don't forget that as
long as changes are not pushed upstream, this can all be reverted (see
git-reset(1) and git-revert(1) ).
Next, we need to add the tag in hostmaster, and push it to the d.o repos. So in short, this sums up as:
cd provision
sh release.sh 1.10
cd ../hostmaster
git tag 6.x-1.10
git push origin 6.x-1.10
Notice how we just provide the Aegir release number (1.10
) to the release script, not the Drupal branch (6.x
), which is hardcoded in the script to remove potential confusion.
Special, for 2.x: While we wait for core support in drush (issue #1991764) we need to manually modify the makefiles in hostmaster, drupal-org.make and hostmaster.make to point to the tags we lay down manually in hostmaster, hosting and provision (and maybe eldir). So this ends up being:
cd provision
sh release.sh 2.0
cd ../hostmaster
git pull
git tag 6.x-2.0
git push origin 6.x-2.0
cd ../hosting
git pull
git tag 6.x-2.0
git push origin 6.x-2.0
2.4.1. Optional: new Eldir release
If we do a major release (say a point zero release), we may want to make a new release of the theme (Eldir). This can also be performed if there enough new changes on the theme to warrant a new release on its own.
To do a new release of Eldir:
- tag and push the tag
- update the version to fetch in hostmaster.make
- create a release node for Eldir
2.5. Test the manual install in Jenkins
Before making a full release, test the release in Jenkins. To do so, start a build of the launch the S_aegir_6.x-1.x_install job (or S_aegir_6.x-2.x_install job for 2.x) with the following parameters:
- the right release as the
AEGIR_VERSION
parameter - the
DRUSH_VERSION
to match what is required for this release. (Currently '4.5' for Aegir 1.x)
If the build fails, delete the remote tags (using git push origin :6.x-1.7
, for example), fix the bugs and start again.
2.6. Build the Debian packages
Build the package and upload to http://debian.aegirproject.org/. Jenkins can build and upload a Debian package for you with the S_aegir-debian-official job.
If you need to move the tags again, you will need to clear the testing archive using the R clear repo job, with the testing argument.
You can also build and upload the package yourself as explained in these detailed instructions. We first upload the package to the testing
distribution, and it gets migrated down into stable
after tests.
Special, for 2.x: build the package manually, see detailed instructions. It should be something like:
./release.sh 2.0-rc5
git reset --hard 6.x-2.0-rc5
git-buildpackage -kanarcat@koumbit.org --git-builder=debuild
dput aegir build-area/aegir2-provision_2.0~rc5_amd64.changes
See the detailed instructions for the dput configuration.
2.7. Test the upgrade in Jenkins
Once both of those tasks have executed successfully, you can test the upgrade of the Debian packages by running the following Jenkins job:
http://ci.aegirproject.org/view/Upgrades/job/U%20aegir%206.x-1.x%20deb%2...
(Note that this test is currently non-functional)
2.8. Creating release nodes on Drupal.org
Once the tags are pushed and release notes published, we create a release node with an excerpt of (and a link to) the release notes so that tarballs are created and issue queue versions updated.
This needs to be done in the hostmaster and provision (and hosting and (maybe) eldir for 2.x) projects on Drupal.org.
Note: this could be automated with the right stuff on Drupal.org.
Special, for 2.x: make sure release nodes are created in order! The projects shipped with hostmaster (hosting, eldir, etc) need to be fully released before the hostmaster release node is created!
2.9. Build the release in Jenkins again
At this point, it's possible that the tarballs on Drupal.org were not created properly. We want to test the real procedure, so run a your build again, but choose 'package' as the AEGIR_FETCH_MODE
. S_aegir_6.x-1.x_install job
Note: The link provided may reference the wrong test, since there doesn't appear to be an AEGIR_FETCH_MODE parameter.
2.10. Publish the Debian packages
Finally, when the Debian packages are tested you will need to pull them into the stable release channel:
http://ci.aegirproject.org/view/Release%20scripts/job/R%20repo%20pull/
2.x note: we pull to testing (and stable since the betas), manually:
reprepro copy testing unstable aegir2
reprepro copy testing unstable aegir2-hostmaster
reprepro copy testing unstable aegir2-provision
reprepro copy testing unstable aegir2-cluster-slave
3.x note: we pull to stable (since the betas), manually:
reprepro@zeus:~$ reprepro copy stable unstable aegir3
reprepro@zeus:~$ reprepro copy stable unstable aegir3-hostmaster
reprepro@zeus:~$ reprepro copy stable unstable aegir3-provision
reprepro@zeus:~$ reprepro copy stable unstable aegir3-cluster-slave
If Jenkins has managed to build .debs and upload them before you've have a chance to pull them into testing/stable, you can manually remove them like so:
reprepro remove unstable aegir2-cluster-slave aegir2 aegir2-provision aegir2-hostmaster
This should be prevented by disabling the Jenkins jobs named 'D_aegir-debian*' before starting to push the release tags.
You can then re-upload the new .debs you've generated, using the '-f' (force) flag:
dput -f aegir build-area/aegir2-provision_2.3_amd64.changes
After doing that, you can re-run the 'copy' commands to publish the .debs to the appropriate releases.
2.11. Publish the release notes widely
Once all this is done and the tarballs are generated, the release notes are published in:
- The handbook (this is where the release notes live!)
- A link to the release notes on the frontpage block
- An event in the calendar
- A discussion post (don't forget to make it 'sticky' & remove stickiness from the previous announcement)
- Update the version in the script upgrade page
- The topic of the IRC channel
- The aegir-announce mailing list (announce@lists.aegirproject.org.)
- Twitter as @aegirproject
Optionally, blog posts on koumbit.org, mig5.net, and elsewhere may go into further detail about significant changes, screencasts etc.
Building and working with the debian packages
Debian is one of the main supported operating systems in Aegir. For other systems, see Operating System Support. See also the following instructions:
The following is aimed at developers wishing to maintain their own Debian packages or work within the packaging framework.
1. Basic requirements
You need the following packages to build the Aegir Debian packages:
apt-get install devscripts git-buildpackage
See also the section below on Adding a new uploader.
2. Building a package for a new release
Assuming we have just released 2.0-alpha2, those instructions will merge that code into the upstream
branch (which is used to create the Debian diff) and then merged again in the debian
branch (where the Debian code lives). We then use git-buildpackage
to build the package and tag it, then push those changes back in the repository.
cd provision
git pull
(if you previously ran release.sh)
git reset --hard 6.x-2.0-rc1
(otherwise run this next line)
dch -v 2.0~alpha2 -D unstable new upstream release
git-buildpackage -kanarcat@koumbit.org
dput aegir ../build-area/aegir-provision_2.0~alpha2_i386.changes
Note: notice how the version number is slightly different in Debian - we use the "magic" ~ separator to indicate that 2.0~alpha2
is actually lower than 2.0
...
The packages are initially uploaded to the "unstable" repository for initial test builds. The idea is that this final package can be moved to "testing" for broader testing, using the command:
sudo -u reprepro reprepro -b /srv/reprepro/ copy testing unstable aegir2 aegir2-provision aegir2-hostmaster aegir2-cluster-slave
When confirmed as ready, it is migrated to the stable repository, using the command:
sudo -u reprepro reprepro -b /srv/reprepro/ copy stable testing aegir2 aegir2-provision aegir2-hostmaster aegir2-cluster-slave
3. Building a branch package
Sometimes you want to have a test package for a given branch without going through a full release. Here how it's done.
git checkout debian
git merge 6.x-1.x
git describe
dch -v 1.0~rc3+28-1
git-buildpackage --git-tag -kanarcat@koumbit.org
This is also available in the Debian package as:
./debian/rules jenkins-build-auto
4. Installing packages manually
dpkg -i aegir-provision_1.0~rc3+g6632e6e-1_all.deb
We also make sure our custom makefile fetches the right one from provision:
-includes[aegir] = "http://drupalcode.org/project/provision.git/blob_plain/6.x-1.0-rc3:/aegir.make"
+includes[aegir] = "http://drupalcode.org/project/provision.git/blob_plain/6.x-1.x:/aegir.make"
5. Developing on Debian
To develop third party extensions to Aegir on Debian, it is recommended to install the Debian packages. If you are working on Aegir core, this could be a bit trickier since the files are not where you expect them to be and are not deployed as git repositories however.
You can, however, copy in place a .git directory using the following:
git clone --branch=6.x-1.0-rc7 http://git.drupal.org/project/provision
cp -Rp provision/.git /usr/share/drush/commands/provision/.git
cd /usr/share/drush/commands/provision
git stash
This will bring back a bunch of files that are removed from the Debian package, so it will yield warnings on uninstall of the Debian package but it should otherwise work.
You can do something similar with the frontend.
6. Package versioning
The stable repository should contain the latest release. The testing repository will also contain the latest release (unless we're in the process of building a release) but could have fixes to the Debian package that are being tested. The unstable repository is automatically built from the stable branch and may be broken.
To see what changes are done to the Debian package, see the debian/changelog which is maintained on the debian branch. To see which version of the package is currently available in the repository, you will unfortunately need to parse the Packages file for unstable, testing or stable.
7. Adding a new uploader
To enable a new maintainer to upload to the Debian repository at debian.aegirproject.org, something like the following steps will have to be followed:
Create a ~/.dput.cfg with the following entry:
# See /etc/dput.cf for examples
[aegir]
login = *
# login = another_username
fqdn = debian.aegirproject.org
method = scp
incoming = ~reprepro/incoming
Next, GPG keys will have to be authorized to upload to the repository:
sudo -u reprepro -i
gpg --search-keys foo@bar.com
gpg --fingerprint foo@bar.com ; gpg --check-sigs foo@bar.com # check if this is the real key
echo allow * by key 1234ABCD >> /srv/reprepro/conf/uploaders
8. Replacing an expired key
gpg --gen-key
gpg --list-keys
gpg --keyserver pgp.mit.edu --send-keys <key id>
sudo -u reprepro -i
gpg --search-keys <key id>
gpg --fingerprint foo@bar.com ; gpg --check-sigs foo@bar.com # check if this is the real key
echo allow * by key <key id> >> /srv/reprepro/conf/uploaders
9. How the archive was built
The following documentation was used: https://wiki.koumbit.net/RepreproConfiguration
1.0 roadmap
This is a brief outline of development plans for Aegir.
Update: this roadmap is not up to date anymore. Effectively, 0.4 and parts of 0.5 and 0.6 were all squashed into an API-stable 1.0 release published on Drupal.org in spring 2011. 1.0 is now in maintenance mode and only critical bugfixes will be accepted in the tree. This roadmap is therefore superseded by the 2.0 roadmap.
Note that all formal Aegir releases (ie. not those marked, "alpha", "beta" or "release candidates" or "RC") are production-ready. We strive hard to maintain good code quality and Q/A by frequent releases and alpha/beta/RC stages before those formal releases so even if the releases are marked 0.x, it's not because they are not stable or ready, it's simply because the API is not stable (see below).
0.1 - "Sites" - Single platform support
Solid release for Drupal 5 platform. Will not be able to upgrade. Production ready.
0.2 - "Upgrades" - Multiple platforms support
Support for more than one Drupal release. Support for migration between Drupal releases. Experimental support for remote SQL servers. Frontend still in Drupal 5.
0.3 - "d6" - Drupal 6 frontend support
In the backend: bugfix release of 0.2. In the frontend: port to Drupal 6.
0.4 - "Cloud" - Multiple servers support
Support ssh as communication channel between multiple servers (see this issue) Move sites between servers, and support for multiple db servers and moving between them.
0.5 - "apt-get" - proper deployment procedures
proper deployment procedures (features? proper project management? etc)
0.6 - "UX" - User interface improvements
Clean up and make the user interface friendlier.
1.0 - "API" - API freeze
A stable, published API for Hostmaster modules that will be frozen during the whole lifetime of the 1.0 branch to ease long term support for third party extensions.
From there on...
Extension modules
There is also a host (pun intended) of provisioning tasks that it would be nice to see created:
- LDAP (open LDAP)
- Solr Search (Tomcat)
- Jabber (ejabberd)
- DNS (Bind, DNSmasq etc - Actually already implemented in the 0.4 line)
- Pound (load balancer)
- Email (Zimbra)
- An RDF triple store (needs more research)
- Asterisk (voicemail, proxy)
Mass site operation modules
- Search (could be related to Solr)
- Statistics (info on nodes/comments/users etc. across all / many sites)
- Logging (centralized logging overview, might just be web interface for syslog)
- Spam / Comment moderation (we had some discussion about writing a Mollom module that works across sites)
Feel free to contact the developers if you are interested in pitching in to work on one or more aspects of Aegir.
2.0 roadmap
Following the Drupalcon Chicago 2011 BOFs discussion, the community came up with a series of ideas and wishlist items for the upcoming 2.0 release. This roadmap may be updated or trimmed down as development goes ahead.
We do not commit to doing all of this for 2.0, this is mostly a wishlist of what people expressed were priorities for them during the brainstorm. If people want things to move forward, they will need to submit patches or create feature branches, basically lead development on the feature you want. This page can be used to coordinate overall leadership of those issues, while pointing to the Drupal.org issue queues for the specifics.
This roadmap supersedes the previous 1.0 roadmap.
Update: this roadmap is very ambitious, and should be considered more as a "wishlist". What we agreed on during the february 6th 2012 scrum was that we would focus on those items, in order of higher to lower priority:
- Autoloaded classes (means we drop support for PHP 5.2 and below)
- Drush 5 support
Once all those are done, 2.x should go in beta and once there are no criticals, it gets release, even if we don't have all the other features listed below.
Update: this is superseded by this roadmap: http://community.aegirproject.org/discuss/another-stab-short-and-long-te...
1. Drush collaboration
We want to collaborate more with the good people in the Drush project. We had good conversations at Drupalcon and hope to keep on having more about how to do staging, sharing code, working with contexts and aliases and so on.
This collaboration should permeate the whole 2.0 release cycle, as a lot of goals here overlap work already happening in the Drush project. We have a long tradition of leading development of new features in Drush, by first implementing them in Provision, and we should continue that trend, with a focus on merging those changes back up into Drush, and eventually removing them from Provision in 2.0.
Koumbit plans to keep on taking care of the Drush Debian packages and coordinate with the Drush team as much as possible.
1.1. Sending more code upstream
Our "contexts" code could be refactored and sent upstream. provision-save would be a likely first target, along with the persistent "contexts" classes. Note that we have hit a significant wall in the current API because it is hard to interoperate with existing server and site contexts from the outside of the class, so that's something that should be cleared up before going ahead with those merges.
1.2. Removing common code
At the very least, we should start removing common code between site-install and provision-install. The new drush archive command should also replace our builtin provision-backup, and provision-save may be refactored around drush_save_config() too (see issue #736686 for this).
1.3. Standard archive support
... which brings us to the standard archive format. Drush and Aegir developers have collaborated to create a common format to export and import Drupal sites. A single archive could bundle one or many sites, including site-wide code, so this is a significant difference (and improvement) over the current de-facto Aegir standard.
We want to support that new format. Full multi-site archives should be imported as platforms, while Aegir could keep on exporting single-site archives for standard platforms, like OpenAtrium or other distributions.
1.4. Unit testing
Drush has started to implement unit testing with PHPUnit in Drush 5 (see the tests directory for samples), and we should jump in that party, extending those classes and unit-testing the hell out of Aegir itself. That will make merging features into Drush easier and make sure we have no regressions while doing so.
2. Multi-server improvements
While Aegir 1.0 has multi-server support, it is far from ideal or complete, and we wish to harden that design in 2.0.
2.1. From a hub-spoke to a mesh model
The current model where there is one main server (the "hub") where all platforms and sites are created and then synced to remote servers (the "spokes") seems to be showing scalability issues and concerns about where the authoritative data should be.
For example, it's currently not recommended to allow developpers to access sites on remote servers, as the authoritative data resides on the hostmaster site. Another issue is that aliases are maintained only on the master server, and therefore are not accessible on remote servers. Finally, the fact that drush and provision are not installed on remote servers make some operations unnecessarily slow as we need to make the operation locally and then sync the result.
The alternative has yet to be discussed. We especially want to chime in with the Drush community here to see how the best practices should be. The conversation has started, and with the migration of Drupal.org to git, a lot of possibilities just opened up, with a lot of people being all of a sudden more familiar with this excellent too.
In general, it should be possible to expect remote servers to have drush and provision installed and be relatively independent from the "master" server. A design like this could eventually mean that Aegir could manage other Aegir sites and upgrade them. It is the "mesh" design, of which the "hub and spoke" model is only a subset.
This follows the practice of "tools not policy" we are heading towards: we leave the mesh vs hub/spoke decision to the site administrators and provide the tools to support both approaches the best we can.
Koumbit plans to spearhead this effort this summer.
2.2. Improve cluster support (done & backported to 1.7)
The current multi-server model doesn't support clusters and shared filesystems very well. Aegir 2.0 should allow scaling your web hosts horizontally by spreading a single site amongst multiple servers. While 1.0 can setup sites on remote servers, it follows more the concept of "sharding", spreading different sites amongst different servers than load balancing all sites amongst all servers.
This involves supporting load balancers (e.g. IPVs, Pound, Varnish), maybe through the existing "cluster" service, but also shared filesystems (e.g. NFS, OCFS, GFS, GlusterFS), maybe through a new "fileserver" service.
Koumbit plans to spearhead this effort this summer.
Update: this has been implemented with the "web_pack" module, which will be released with 1.7, deprecating the cluster module. 2.x will remove support for the current "cluster" module. See also the updated documentation.
2.3. Optimizing clones and migrates
We quickly mentionned git support above, but this could lead to significant improvements, for example by using git clone --shared
instead of the traditional "backup and deploy" approach to clone/migrate sites.
There is certainly the possibility of rewriting clone to optimize away "backup and deploy" and merge them in a single operation that could use copy on write mechanisms like hardlinks (cp -al
), git tricks (git clone --shared
) or filesystem snapshots (e.g. BTRFS snapshot directory/file support).
Once deploy is optimized, we could consider hooking it the Drupal 7 or third party unit tests to rollback on such failures.
Those optimizations would need to integrate with and support the multi-server model elegantly too, especially the hypothetical fileserver service.
3. Modularization of more systems
Aegir 1.0 is already pretty modular: "services" are an abstract concept that can be implemented and inherited. For example, the "HTTP service" has four implementations: Apache, Apache-SSL, Nginx and Nginx-SSL. The database and DNS layers are also abstracted that way.
But more can be done here.
3.1. Modular backup system
Right now backups can be pretty heavy and use the venerable (1979!) tar backup format and utilities, which made Aegir portable to almost any UNIX operating system. It's a good start, but it's highly inefficient. As mentionned above for the clone and migrate optimization, we could also use modern filesystem features like snapshots or use git support. Another suggestion was to use S3 or similar services for offsite backups.
The basic idea here is that backups should be extendable - if you want to backup in your shoebox, Aegir should allow you to write a plugin for it. There was some initial work done by computerminds on github.
3.2. Modular platform support
This is also known as the alien platform support, that is - being able to install other "things" than Drupal in the backend, for example (oh the heresy!) Wordpress. This involves ripping apart the "platform" code in the frontend and the backend to make it more CMS-agnostic, so that Drupal is just one of the implementations.
This would make sense of course if at least another implementation would be supported. This could also involve merging the site and platform creation forms for those platforms that do not support multi-site like Drupal does (so that creating a site effectively create a platform).
3.3. Modular queuing system
The current queuing system is what we informally call the "ghetto queue". It's a cron job that pulls tasks from a MySQL database. While this scales well for a few sites, on busy servers it's a total nightmare, especially for mass migrations. Tasks stack up and concurrency is not managed at all. This needs to be completely redone.
A possibility is to use Drupal 7's abstracted queue API and let administrators implement their own queuing system according to their local APIs.
The new queuing system could be a key part in how the multi-server "mesh" model would be implemented. It could also be how non-Drush backends could be handled by Aegir - in that scenario, Aegir would become an interface for a standard queuing system, yet to be determined, although a lot of ideas are flying around: Jenkins/Hudson, AMPQ, Drush itself, etc. See this evaluation for that decision.
Koumbit plans to spearhead this effort this summer.
4. Other features
4.1. Automatic upgrades
There were a few discussions around managing platforms automatically. The implementation of hosting-import is one of the building blocks of this, but there remains some fine-tuning for this to work out of the box, especially for new users, some of which have requested that we can build makefiles straight from the frontend, a bit like how it's done on drushmake.me. One could also use feature servers feeds to update their platforms. Again, we should leave the policy to the user or at least make this pluggable.
This mostly consists of automating the creation of new platforms, the upgrade of sites between platforms and garbage collection of old platforms.
4.2. PostgreSQL backend support
The backend database layer (the "db class") now supports only MySQL, that is, while the Aegir frontend could technically run in PostgreSQL (although that is still unsure), the backend can't create sites in a PostgreSQL database.
Most of the work here will be around supporting user credentials handling, creation and destruction, which are fairly different between postgres and MySQL. The issue for this is #585796.
4.3. Statistics improvements
Work has already started at Koumbit to implement a "statistics" queue that would mostly pull Apache logfiles and parse it for size (and CPU timing) information. This is to implement bandwidth and CPU quotas, but could be extended in a more generic "statistics" queue that would collect all sorts of information:
- if your Drupals need to be upgraded or are secure (the basic "status report")
- how many modules/themes/profiles/libraries are installed (populating the frontend data)
- how many users/nodes/fields instances do you have?
All this could be collected in a nice dashboard that would outline client, site or platform-specific information.
4.4. Full DNS support
The DNS support in the backend is already pretty mature, but it's not really intuitive how it works and how to integrate it into existing infrastructures. Issue #922280 details all the next steps required here, but we can already say we hope to have a zonefile editor in the frontend, probably based on D7 entities, reasonable access control for zones, registrar support and "host my mail on gmail button" type of wizards.
This could potentially be picked up by a GSoC student.
4.5. Improved intersite security
We are already going through some hoops to make sure the database credentials are not accessible from one site to the other. The credentials are stored in the Apache environment and cleared out as soon as they are used, which makes finding them in settings.php impossible. Koumbit has also worked on a Provision ACL module that implements POSIX ACLs on files which allow more granular access controls on files.
But there are other security issues remaining that we should be looking into, especially the way Drush and Drupal are bootstrapped during tasks, which is currently insecure.
Koumbit plans to spearhead this effort this summer.
5. Contrib modules
Other contrib modules may have different roadmaps, but they must respect the release naming conventions. Contrib modules should aim for a 1.y release now, that will be compatible with the 1.x series. 2.x modules should be ported to Drupal 7 and follow the API (in development) of the 2.x series. Modules do not have to follow the minor version numbers of Aegir. For example, Provision ACL 1.1 is compatible with any release of the 1.x core series.
We are also considering making point releases of the backend and the frontend independent, so that Provision 1.1 and Hostmaster 1.2 could be release independently.
5.1. Other modules roadmaps
This is a short index of other modules roadmaps to 2.0:
D7 port & DNS editor plan (google summer of code 2011)
Plan for porting Aegir to D7 and creating a DNS editor. A project for Google Summer of Code 2011.
Please offer advice on how to improve the plan I've outlined below.
Relevant links: Hostmaster sandbox for GSOC2011
Before May 24:
- Review Git docs and other resources to learn more about using Git.
- Review issues/patches relevant to D7 port and DNS editor.
- Review api.aegirproject.org and community.aegirproject.org/developing.
- Discuss planned work with aegir maintainers.
- Refine plan below based on what I've learned from the handbook, api, and aegir maintainers.
Week one: Work on porting Aegir theme Eldir to Drupal 7. Finalize plans for porting the Aegir modules and install profile.
Week two: Finish porting Eldir to Drupal 7. Port the Aegir Hosting module to Drupal 7.
Weeks three through six: Port the Task, Client, DB Server, Package, Platform, Site, and Web Server modules to Drupal 7. Test ported modules.
Weeks six through nine: Port modules Alias, Clone, Cron, Migrate, Quota, Signup, Task, and Web Cluster to Drupal 7. Test ported modules.
Weeks ten and eleven: Work on the DNS zonefile editor.
Weeks twelve and thirteen: Continue testing ported projects and the DNS zonefile editor. Update documentation to reflect changes in the upgrade to Drupal 7.
Weekly Scrums
Since we have now a bigger team of people interested in the project (contributors in the queue, outside the queue, people at Koumbit, the community site, my my!!), we (koumbit) feel it would be important to start doing weekly scrums again.
Nothing complicated: just a 15 minutes where we agree we try to be available on IRC to talk about various issues that have or will come up during the week. No work, just talk.. to quote wikipedia:
(It is the role of the ScrumMaster to facilitate resolution of these impediments. Typically this should occur outside the context of the Daily Scrum so that it may stay under 15 minutes.)
- The meeting starts precisely on time.
- All are welcome, but only “pigs” [devs] may speak
- The meeting is timeboxed to 15 minutes
- The meeting should happen at the same location and same time every day
- During the meeting, each team member answers three questions:
- What have you done since yesterday?
- What are you planning to do today?
- Do you have any problems preventing you from accomplishing your goal?
These occur via IRC on the #aegir channel twice a week at separate times:
Every Monday at 20h00 UTC. This works out to be:
- 11h00 in Los Angeles, USA (PDT)
- 15h00 in Montreal, Canada; New York, USA (EDT)
- 19h00 in London, UK (GMT)
- 06h00 in Melbourne, Australia (EST)(on Tuesday)
(See this useful tool for how we settled on those times.)
Every Thursday at 10h00 UTC. This works out to be:
- 03h00 in Los Angelas, USA (PDT)
- 06h00 in Montreal, Canada; New York, USA (EDT)
- 11h00 in London, UK
- 12h00 (noon) in Warsaw, Poland
- 20h00 in Melbourne, Australia (EST)
We will keep logs of these here for future reference.
Archive: 2010
Archive of the weekly scrum from 2010
Weekly Scrum IRC Log: 2010-12-02
(03:00:54 PM) anarcat: hi. (03:01:08 PM) anarcat: so now is the time for our newly established weekly scrums :) (03:01:24 PM) anarcat: of course, mig5 is not there and vertice either, but i'll still do my part, just to kickstart this (03:01:32 PM) hadsie: hey anarcat (03:01:54 PM) anarcat: hey hadsie (03:03:30 PM) anarcat: so basically, i haven't done much wrt to aegir this week, i'm busy with other koumbit work (03:03:39 PM) anarcat: you're going to hear this often from me, unfortunately :) (03:04:03 PM) anarcat: my latest thing though is the creation of a debian package for provision that can install and upgrade aegir automagically, it's pretty neat and needs testing (03:05:42 PM) anarcat: i have contacted other debian developpers to get their feedback about the possibility of uploading that package straight into debian (03:07:24 PM) anarcat: the followup for the debian package happens here: http://drupal.org/node/490004 and here: http://bugs.debian.org/532923 (03:07:36 PM) sfyn: hey folks (03:07:36 PM) anarcat: in other news i started looking at STS here: http://drupal.org/node/986312 (03:08:04 PM) anarcat: so that's it for my past work, in the next week i'll try to: (03:08:22 PM) anarcat: figure out proper permissions for multi-(shell)user setups in aegir (03:08:49 PM) anarcat: right now a lot of our devs use sudo -u aegir to do stuff and that's quite messy, we're going to try to figure out the minimum set of perms for that to work, maybe with ACLs (03:09:03 PM) anarcat: we also need to figure out how we maintain site-specific packages in remote servers (03:09:26 PM) anarcat: and i want to start fixing DNS in the frontend, but we are slowly pushing back our objective of having DNS in production to after christmast (03:09:30 PM) anarcat: so that's it for me (03:09:42 PM) anarcat: hi sfyn :) (03:09:43 PM) anarcat: sfyn / ergonlogic / mvc : anything to add? (03:11:24 PM) simesy: hey i actually made the scrum, (but only because I had to get up for a job and now I have to go :P) (03:11:35 PM) anarcat: :) (03:11:35 PM) anarcat: i see (03:11:39 PM) anarcat: well, congratulations :) (03:11:42 PM) sfyn: I have been sick all week and trying to catch up on my rpod stuff (03:11:51 PM) sfyn: So very little progress to report (03:11:59 PM) sfyn: I have a guy who is interested in writing tests for us (03:12:13 PM) sfyn: Luigi - are you here? (03:12:38 PM) sfyn: A guy in montreal, but he is a little new to the fabulous world of drupal, and FLOSS in general, I get the impression (03:12:46 PM) sfyn: But he has done unit testing work before (03:13:00 PM) sfyn: Maybe we should get him to come into the office so we can orient him (03:13:18 PM) sfyn: hadsie: hey man (03:14:07 PM) anarcat: thank you sfyn (03:14:16 PM) anarcat: so our little scrum is almost over, anybody got anything to add before we move on? (03:16:26 PM) anarcat: thank you for your attention, and see you next week!
Weekly Scrum IRC Log: 2010-12-09
(02:57:33 PM) ergonlogic: everyone: Weekly scrum being in ~5 mins (02:57:57 PM) mig5: trying to wake up :) (03:01:49 PM) anarcat: hey hey! (03:01:55 PM) anarcat: good morning world! (03:02:01 PM) omega8cc: hey (03:02:10 PM) ***anarcat tries to play the gooooood moooorning vietnam cue here (03:02:12 PM) mig5: good morning, here at least :) (03:02:16 PM) anarcat: so (03:02:31 PM) anarcat: welcome to our weekly scrum session, we have ... 13 minutes left (03:02:35 PM) mig5: woot (03:02:41 PM) anarcat: we failed at the first directive, and i'm sorry for that (start on time, damnit!) (03:02:50 PM) ergonlogic: my fault (03:02:51 PM) anarcat: and i'll start with my stuff, i didn't do much last week (03:03:02 PM) anarcat: in fact, nothing i can remember (03:03:14 PM) anarcat: next week is likely to be similar as i'm over[load]ed with thousands of internal needles (03:03:20 PM) anarcat: that's all for me :) (03:03:22 PM) mig5: yay (03:03:29 PM) mig5: ok (03:03:47 PM) mig5: to carry on that tune, i didn't do much last week, i have been trying to catch up on some tickets (03:03:53 PM) mig5: my immediate aims are, (03:04:04 PM) mig5: to test the debian packages now that i know they're in unstable (03:04:07 PM) mig5: which i will do today (03:04:24 PM) mig5: and possibly help ergonlogic with this clients permission stuff (03:04:45 PM) mig5: after that i will start looking at the 'for review' patches in the queue per our beta2 plans (03:04:51 PM) mig5: but that may not get to happen until next week (03:05:01 PM) anarcat: rc2 you mean (03:05:02 PM) mig5: that's about it from me (03:05:06 PM) anarcat: ok (03:05:08 PM) mig5: oh really (03:05:10 PM) mig5: rc2? (03:05:12 PM) anarcat: er (03:05:13 PM) anarcat: no (03:05:15 PM) anarcat: i don't (03:05:16 PM) mig5: haha (03:05:18 PM) ***anarcat is on crack, nevermind (03:05:24 PM) anarcat: i was confusing us with drush, sorry (03:05:29 PM) mig5: i would be happy to push for rc1 :) (03:05:32 PM) anarcat: ergonlogic? (03:05:48 PM) anarcat: sfyn is offline today, but i know he tried to get back into things and will keep on doing that (03:05:50 PM) ergonlogic: well, I'm working on the permission issue that mig5 mentioned (03:06:02 PM) ergonlogic: and the udev dependency (03:06:11 PM) anarcat: right, about that (03:06:22 PM) anarcat: i'm sure that moshe_work and other drush people would love to see that land in drush 4 (03:06:27 PM) ergonlogic: just trying to get into the swing of things, and familiarize myslef with some of aegir's inner workings (03:06:32 PM) anarcat: but i'm not sure we'd make it for rc3 at this rate (03:06:51 PM) anarcat: how do you feel about this? (03:07:32 PM) ergonlogic: well, I think I'm about half way there on that issue (03:07:40 PM) anarcat: alright (03:07:41 PM) ergonlogic: but I still nee to post a patch (03:07:47 PM) ergonlogic: s/nee/need (03:07:48 PM) anarcat: ok (03:08:11 PM) anarcat: *and* it would then need to be ported up to drush... so we can safely assume we won't make it into drush 4, those guys are in a rush :) (03:08:20 PM) anarcat: speaking of drush (03:08:28 PM) mig5: in a rush or in a drush? :) (03:08:33 PM) anarcat: drush is that somebody omega8cc tested aegir with drush 4-rc1, and it works (03:08:34 PM) anarcat: hehe (03:08:35 PM) ergonlogic: groan (03:08:40 PM) mig5: sorry.. (03:08:44 PM) anarcat: hehe (03:08:49 PM) anarcat: alright (03:08:55 PM) mig5: yes Vertice asked me to test that too, another thing on my todo list (03:08:59 PM) anarcat: so (03:09:00 PM) mig5: glad someone else did (03:09:02 PM) anarcat: i guess one last thing (03:09:32 PM) anarcat: we heard from vertice finally and he's coming back after a small hiatus, he's going to concentrate on documentation stuff and helping us figuring [stuff] out (03:09:40 PM) anarcat: he's also eager (hehe) to release an rc (03:09:45 PM) anarcat: so maybe it's something we should consider (03:09:49 PM) ***anarcat looking at rc bugs (03:10:22 PM) mig5: we have had the benefit of not having too many new criticals. i think if we can tie off some existing patches, we could look at that perhaps (03:10:26 PM) anarcat: so most of those are postponed (so needs testing and closed: worksforme) (03:10:44 PM) mig5: do you have a link to the rc bugs (03:10:50 PM) anarcat: http://drupal.org/project/issues?text=&projects=provision,+hosting,+hostslave,+eldir,+Hostmaster+(Aegir)&status=Open&priorities=1&categories=All (03:11:05 PM) anarcat: one is the freaky multiserver thing, i'd like adrian or somebody else to look at that one as i don't have the resources or time or brain to look into this (03:11:11 PM) anarcat: that one is http://drupal.org/node/976300 (03:11:20 PM) anarcat: the other one is the site form optimisation: http://drupal.org/node/955854 (03:11:27 PM) anarcat: which could be postponed to post 0.4 (03:11:33 PM) anarcat: so we need testing, basically (03:11:39 PM) mig5: yeah i'm still not convinced the multiserver thing is just someone trying to do something we don't support at all (i.e spoke vs mesh model) (03:11:42 PM) mig5: yep (03:11:49 PM) anarcat: alright (03:12:03 PM) anarcat: so i would favor more a testing + rc1 than a beta2 at this point (03:12:07 PM) ergonlogic: I'm into testing the multi-server stuff, if that's on the menu (03:12:17 PM) anarcat: ergonlogic: it is, take a look at http://drupal.org/node/976300 (03:12:20 PM) ergonlogic: and dns (03:12:29 PM) anarcat: dns + ssl integration needs testing too: http://drupal.org/node/934908 (03:12:44 PM) anarcat: and this should probably just be closed: http://drupal.org/node/950118 (new servers probably work fine) (03:12:52 PM) anarcat: that one is more hairy: http://drupal.org/node/884090 (03:12:57 PM) anarcat: not suer about it (03:13:12 PM) anarcat: but really, i believe we could just test those quickly and release next week, how about that? (03:13:56 PM) mig5: sounds good, perhaps late next week, allowing me my thursday/friday where i can spend like a whole day looking at some of this (03:13:56 PM) ergonlogic: I'll look into provisioning some new servers :) (03:14:02 PM) anarcat: ok (03:14:04 PM) anarcat: awesome (03:14:07 PM) anarcat: okay, anybody else has something to add here? (03:14:24 PM) anarcat: so thank you for attending, and have a nice day! (03:14:31 PM) ergonlogic: cheers! (03:14:34 PM) mig5: cheers
Weekly Scrum IRC Log: 2010-12-16
07:00 <@mig5> scrum time 07:00 <@anarcat> hi 07:00 <@mig5> morning 07:01 <@anarcat> good morning! 07:01 <@anarcat> not much to report on my side again, other than i forgot to book a session for drupalcon :/ 07:01 <@mig5> hehe 07:01 <@anarcat> we're working on LDAP support and we'll focus on intersite security after christmas 07:01 <@mig5> i did make a tweet after i saw there are three aegir related proposals 07:01 <@mig5> oh sweet! LDAP++ 07:01 <@anarcat> add a way for aegir to run tasks as a real user instead of itself, so that site A can't see site B's stuff 07:03 <@mig5> cool - that's it from you? 07:04 <@anarcat> yes, sorry 07:04 <@mig5> no worries 07:04 <@mig5> ok, so (again) i haven't done much 07:05 <@mig5> i tried to address some of the for review tickets in the queue, committing two of hadsie's work which i tested 07:05 <@anarcat> that's much :) 07:05 <@mig5> i didn't get to test your debian packages against another squeeze install but i did comment on the ticket about what i think the issue is there 07:05 <@anarcat> yeah, you're right about the issues, feel free to fix if you have time 07:06 <@mig5> but mainly i have been just staring at the queue and feeling out of steam (again), I wonder whether it is time to look at adding more people to the project 07:06 <@anarcat> i agree 07:06 <@mig5> as i feel i'm getting less and less useful, and vertice is taking a break etc, 07:06 <@mig5> and you are tied up a lot with internal stuff (which is fine) 07:06 <@anarcat> i don't feel you're less and less useful though :P 07:07 <@mig5> well that's ok :) i think it speaks for itself 07:07 <@mig5> and things have slowed down so lets get some more hands on deck basically. 07:07 <@mig5> also, i have a client at the moment who is interested in seeing two things happen: 07:07 <@mig5> a dns frontend and scheduled tasks 07:07 <@mig5> Steven Jones has been doing great work with scheduled tasks and created a contrib project hosting_backup_queue 07:08 <@mig5> i was going to ask whether Koumbit were already working on a dns frontend 07:08 <@mig5> we can discuss that later if you like 07:08 <@mig5> and that's about it from me 07:09 <@anarcat> ok, anybody else? 07:09 <@mig5> maybe it would be nice to summarise these scrum sessions as a 'weekly aegir news' email to the mailing list, making people aware of stuff like Steven Jones' contrib project etc 07:09 <@anarcat> mig5: i have ideas on the frontend: 1) there should be code we can resurrect from older git repos and 2) i have a workflow for the blacklisting algorithm (can't let anyone host any zone can't we...) 07:09 <@anarcat> yep 07:10 <@mig5> i am happy to do so 07:11 <@mig5> (re: news) 07:11 <@mig5> ok -we are 11 minutes in anyone else have anything to add? 07:12 < erutan> re: news to save time you could always just dump the relevant irc log onto the open atrium site 07:12 <@mig5> we do that already, yep 07:12 <@mig5> maybe that's enough. 07:12 <@anarcat> yeah... 07:13 <@anarcat> alright, thanks for coming everyone! 07:13 <@mig5> cheers 07:13 <@mig5> cheers 07:14 <@mig5> adding the irc log now 07:15 < noecc> mig5: anarcat: vertice: and others You have done great things with aegir. Don't let your steamlessness get you down. 07:15 <@anarcat> :) 07:15 <@anarcat> thanks :)
Weekly Scrum IRC Log: 2010-12-23
Note that none of the core devs were in attendance for this session. Regardless, this is the discussion that ensued:
(03:01:56 PM) darthsteven: Is the scrum at 2005? (03:05:10 PM) ergonlogic: I guess I could start, pending the arrival of the core devs (03:05:28 PM) ergonlogic: This week I upgraded to beta2 and began testing remote servers (03:05:58 PM) ergonlogic: the beta2 upgrade required debugging some misconfiguration I had done previously, but otherwise everything seems to be working fine (03:06:32 PM) ergonlogic: so far, I can verify a remote platform, but can't yet provision a site on it (03:06:48 PM) ergonlogic: so that'll be something I look at over the holidays (03:07:03 PM) ergonlogic: finally, I published and api site (03:07:27 PM) ergonlogic: currently at api.ergonlogic.net, but hopefully api.aegirproject.org will point ot it soon (03:07:34 PM) ergonlogic: that's it for me (03:07:49 PM) ergonlogic: happy holidays everyone (03:08:07 PM) eft_: I started a Why Aegir wiki http://community.aegirproject.org/node/217 (03:08:35 PM) eft_: hope to add some graphics at some point (03:08:50 PM) eft_: we need to market Aegir more :) (03:09:50 PM) ergonlogic: eft_: agreed about the mktg, but the move to beta seems to have helped already (03:10:35 PM) eft_: how so? do you mean the perception of getting out of alpha? (03:11:21 PM) omega8cc: I submitted my first attempt to resolve the issue with remote files being removed on site verify, testing it is welcome: http://drupal.org/node/976300#comment-3855932 (03:13:15 PM) omega8cc: the issue is mentioned on http://community.aegirproject.org/0.4-beta2 (03:14:00 PM) ergonlogic: eft_: yes (03:14:31 PM) omega8cc: oh, and www.drupal7releaseparty.org is hosted on Aegir (03:14:52 PM) eft_: cool
Archive: 2011
Archive of the weekly scrums from 2011
Weekly Scrum IRC Log: 2011-01-07
07:01 <@mig5> Scrum time 07:01 <@mig5> who is here for the meeting, anarcat? 07:01 <@mig5> nope :) 07:01 <@mig5> sfyn: 07:02 <@mig5> well i'll start. another week gone by without looking at DNS, but we have a good roadmap now thanks to anarcat, i may start looking at that today 07:03 <@mig5> we're now compatible with drush 4 (at the moment) 07:03 <@mig5> i cleaned up some of the hostmaster profile thanks to drush 4's supporting of drupal_set_message being sent to stdout during install profile execution, which identified some legacy stuff that could be removed 07:04 <@mig5> i'm going to look at dns, upgrades, and some multiserver stuff today hopefully 07:05 <@mig5> i also hope to look at darthsteven's new contrib module hosting_backup_gc http://drupal.org/project/hosting_backup_gc 07:05 <@mig5> and am trying to think of a nice way to let people list their contrib modules on teh community site. i wonder if we should have an inbuilt Featureserver for that sortof thing to let people upload them 07:05 <@mig5> thats it from me (i am talking to myself anyway? :) ) 07:06 <@mig5> i'll leave it for another 5 min and if nothing happens, will drop that into the logs
Weekly Scrum IRC Log: 2011-01-14
06:59 <@anarcat> SCRUM START NOW! 06:59 <@anarcat> welcome everyone :) 06:59 <@anarcat> i'll start with myself 06:59 <@anarcat> i am just out of a lenghty (2.5 week) vacation 06:59 <@anarcat> which was very good 06:59 <@anarcat> i have done absolutely no aegir work since decembre 07:00 <@anarcat> and i don't plan on doing anything until monday 07:00 <@anarcat> at which point we'll update our prod server to head and try to make it work 07:00 <@anarcat> i had big unexpected issues last time with that http://drupal.org/node/1003700 07:00 <@anarcat> in general, i feel there's something very fishy with aliases, i feel this affects more than one of my server instances\ 07:00 <@anarcat> i suspect it may be related to drush 3.3 or the drush package of 07:01 <@anarcat> so i'll be working on that 07:01 <@anarcat> i have also a huge backlog of internal aegir issues that may yield a few bugfixes 07:01 <@anarcat> this will be a good Q/A to maybe lead to an RC somewhere next week 07:01 <@anarcat> so that's it for me 07:01 <@mig5> nice one 07:02 <@mig5> i've not done much except, updated our makefiles to depend on drush 4 and new drush make 07:02 <@mig5> i also updated the upgrade script to remove some hardcoded crap and be more consistent with the install.sh 07:02 <@anarcat> coool 07:02 <@mig5> i *wanted* to work on DNS but sorry, I am just really struggling to do anything aegir :( 07:02 <@anarcat> you've actually done some real work, crazy ;) 07:02 <@mig5> well not really.. :s 07:03 <@mig5> finally, i'll just note on what i think is our most critical bug right now, and i hope to focus on this instead 07:03 <@mig5> http://drupal.org/node/976300 07:03 <@anarcat> Vertice: anything to add? how is the DNS change coming up? 07:03 * anarcat takes a look 07:03 <@mig5> verify a remote platform/site and it deletes shit in your site's /files dir :s 07:03 <@anarcat> right, ok 07:03 <@anarcat> mig5: i think it's good to sync only some files and not the whole dir 07:03 <@mig5> because we do these crazy syncs of the entire codebase *after just syncing the settings.php, and then the drushrc.php, etc* 07:03 <@mig5> yes 07:04 <@mig5> anyway, can discuss that later, just wanted to mention what i think i'll focus on for the moment 07:04 <@anarcat> that's very good. 07:04 <@mig5> and that's it from me 07:04 <@anarcat> and it it's RC indeed 07:04 <@anarcat> i need Vertice to do changes to the DNS to activate ergonlogic's API site so we have api.aegirproject.org 07:04 <@anarcat> anyone else wants to add something? 07:04 <@mig5> i'd like to add to that and possibly take over the hosting of aegirproject.org at least, so we can get that updated 07:05 < ergonlogic> thanks anarcat 07:05 <@mig5> or possibly even take over the DNS for aegirproject.org itself, in the event i ever need to move the mailing list server or something 07:05 < eft_> mig5 : +1 to that idea 07:05 <@mig5> just for the sake of speed, but it's up to devseed 07:05 <@anarcat> yeah 07:05 <@anarcat> i don't mind you hosting that, koumbit can also host DNS and the site 07:05 <@mig5> yeah either way :) 07:05 <@anarcat> i offered this to Vertice, i expect us to have DNS soon 07:05 <@mig5> cool 07:05 <@anarcat> (us = koumbit) 07:05 <@mig5> yep 07:06 <@anarcat> but for the site, yeah, it would be important to at least have a backup ... 07:06 <@mig5> darthsteven: feel free to chime in - you've been doing great work with contrib aegir stuff 07:06 < darthsteven> ah okay 07:06 < eft_> either way, it would be nice to increase the cookie timeout on community.aegir.org 07:06 <@anarcat> yeah, if anyone else wants to jump in, feel free to talk :) 07:06 < darthsteven> So I've been thinking mostly 07:06 <@anarcat> if you're doing dev on aegir, now's the time to talk (after darthsteven :) 07:06 < darthsteven> Mainly about provision contexts 07:07 < darthsteven> and provision-save and services and how all the pieces fit together 07:07 < darthsteven> I have an outline for a handbook page(s) that should document most of what's going on 07:08 <@mig5> oh that'd be nice. since you have chosen the bit that no-one understands how it really is meant to work :) 07:08 < darthsteven> Nothing new in contrib land from me this week 07:08 <@anarcat> cool 07:08 <@anarcat> i think Vertice did some work in that direction in the handbook already 07:08 <@anarcat> ok 07:08 <@anarcat> anyone else? 07:08 < darthsteven> mig5: yeah, it is complex... 07:08 < darthsteven> that's it from me 07:08 <@mig5> yes darthsteven http://community.aegirproject.org/node/41 have a read of that if you've not already 07:09 <@anarcat> cool 07:09 <@anarcat> ok last call, anyone else? 07:09 <@anarcat> anybody have time to log this into the community site? 07:09 <@mig5> sure 07:09 <@mig5> thanks all! goodscrum 07:10 <@anarcat> thanks! 07:10 <@anarcat> happy new year everybody, and thanks for attending! 07:10 <@anarcat> SCRUM END! 07:10 < darthsteven>
Weekly Scrum IRC Log: 2011-01-20
(03:00:17 PM) anarcat: welcome to our weeeeekly scrum! (03:00:28 PM) anarcat: good morning, good evening, wherever you are! (03:00:39 PM) anarcat: those new to the scrum can read on them here; http://community.aegirproject.org/scrums (03:00:43 PM) anarcat: otherwise i'll start (03:00:57 PM) anarcat: yesterday i upgraded our prod server from alpha14 to beta2, and it went well (03:01:00 PM) anarcat: mostly without an hitch (03:01:05 PM) anarcat: i was really surprised, actually (03:01:15 PM) anarcat: i expected to see the weird issues with aliases (03:01:27 PM) anarcat: http://drupal.org/node/1003700 (03:01:30 PM) anarcat: but i didn't (03:01:34 PM) mig5: pretty smooth upgrades from the later alphas (03:01:44 PM) anarcat: however, when upgrading a client from alpha6, i had issues with the upgrade path (03:01:56 PM) anarcat: http://drupal.org/node/973826 (03:02:08 PM) anarcat: the above contains a nice demonstration on how to fix the issues (03:02:22 PM) anarcat: i am not sure we should fix anything there, as it's really hairy to fix and affects very few people (03:02:25 PM) anarcat: not worth it, i think (03:02:32 PM) anarcat: i have filed a bunch of bug nevertheless (03:02:36 PM) anarcat: after our prod upgrade (03:02:40 PM) anarcat: had performance issues with the migrate page (03:02:51 PM) anarcat: which we fixed: http://drupal.org/node/1033072 (03:02:59 PM) anarcat: that needs testing, but maybe I can just merge it now (03:03:13 PM) anarcat: we also have numerous issues with the site form that remain (03:03:15 PM) anarcat: esp http://drupal.org/node/1033078 (03:03:23 PM) anarcat: ie. the ssl cert selection doesn't pop up (03:03:37 PM) anarcat: and that page is still slow as hell (03:03:41 PM) anarcat: which may be related to http://drupal.org/node/955854 (03:03:59 PM) anarcat: so that's it for last week (03:04:06 PM) mig5: wow! :) (03:04:09 PM) anarcat: next week i'll try to fix a shitload of bugs i have in the backlog (03:04:15 PM) anarcat: that other koumbit workers reported (03:04:24 PM) anarcat: mostly upgrade issues, sometimes contrib-modules (e.g. date) related (03:04:27 PM) ergonlogic: anarcat++ (03:04:38 PM) anarcat: once i'm through this, i am thinking of looking again at a RC release (03:04:45 PM) mig5: anarcat has taken over skynet again (03:04:49 PM) anarcat: but i am curious what others think about this (03:04:58 PM) anarcat: mig5: i wish... (03:05:07 PM) anarcat: i still struggle to free some time for aegir here (03:05:12 PM) anarcat: but that may get better with time (03:05:17 PM) anarcat: my focus right now is stabilising our thing (03:05:24 PM) mode (+v grugnog_laptop) by ChanServ (03:05:30 PM) mig5: i am the opposite, i have the time but not the skills :) (03:05:33 PM) anarcat: targeting feb. for stabilisation, then working on security during feb. (03:05:39 PM) mig5: yes for RC, that's a good idea (03:05:49 PM) mig5: i even forgot we agreed to jump from 0.4 to 1.0 (03:05:52 PM) mig5: in september (03:06:01 PM) anarcat: re. taht, we still have 2 RC bugs (03:06:04 PM) anarcat: http://drupal.org/node/1003908 (03:06:07 PM) anarcat: http://drupal.org/node/993944 (03:06:10 PM) anarcat: The bulk operations form for site listings shouldn't list install and import tasks (03:06:14 PM) anarcat: ssl rollback failure (03:06:25 PM) anarcat: we should consider the option of rolling back the bulk ops shit if it breaks stuff (03:06:32 PM) anarcat: but that seems like an easy thing to fix (03:06:36 PM) anarcat: the ssl rollback sounds more hairy (03:06:46 PM) anarcat: but i think we can ship an RC with it as a known issue (03:06:51 PM) mig5: yeah.. i opened a ticket here and there about bulk ops. not really major bugs, just little niggly things (03:06:53 PM) anarcat: that's it for me (03:06:55 PM) mig5: yeah (03:07:01 PM) mig5: ok (03:07:09 PM) anarcat: go mig5 ! (03:07:13 PM) mig5: in classic mig5 style i have done nothing (03:07:15 PM) mig5: EXCEPT (03:07:18 PM) mig5: http://drupal.org/node/976300 (03:07:26 PM) mig5: i fixed our major and possibly only data-loss bug (03:07:39 PM) mig5: that was deleting stuff from a site's 'files'dir on a remote server on every verify (03:07:43 PM) mig5: and hence migrate, etc. (03:07:52 PM) anarcat: "just that" (03:07:56 PM) anarcat: dude that's awesome (03:08:00 PM) mig5: that was a major aegirwtf but it seems to be ok now (03:08:07 PM) omega8cc: that fix makes rc possible, imo (03:08:08 PM) anarcat: cool (03:08:32 PM) anarcat: right (03:08:36 PM) anarcat: mig5: what's up next week? (03:08:43 PM) mig5: i don't think i've done anything else :s i have given up on the idea of getting anywhere with the dns stuff, so i won't make promises there (03:08:44 PM) anarcat: dns? ;) (03:08:47 PM) mig5: haha (03:08:47 PM) anarcat: Vertice: you there? (03:08:51 PM) mig5: damn (03:08:52 PM) anarcat: hehe (03:08:55 PM) anarcat: it's okay (03:09:05 PM) anarcat: our discussion helped me rehash the roadmap, so it's all good (03:09:39 PM) anarcat: mig5: anything else? (03:09:53 PM) mig5: this is weird: http://drupal.org/node/1025972 and i think i've reproduced it, so i will probably take a look at that (03:10:06 PM) mig5: it's not *too* critical but it means remote servers need a manual apache restart or something, after the task completes (03:10:14 PM) anarcat: right (03:10:15 PM) anarcat: ok (03:10:15 PM) mig5: so this is just stability stuff (03:10:22 PM) mig5: that's it from me (03:10:28 PM) anarcat: i have seen weird ordering issues like this even in local, but i never found it to be a bug so far (03:10:31 PM) anarcat: okay (03:10:34 PM) anarcat: omega8cc / ergonlogic / others: any comments? what you did last week / will do next week? (03:10:37 PM) anarcat: Vertice: ? (03:11:10 PM) darthsteven: I didn't get chance to write up 'contexts' this week (03:11:19 PM) darthsteven: But hope to next week (03:11:23 PM) anarcat: awesome (03:11:27 PM) anarcat: otherwise i want to say that we (the aegir project) are always looking for new blood (03:11:34 PM) darthsteven: That's it from me (03:11:49 PM) mig5: nice darthsteven, i am looking forward to learning how this stuff works :) (03:11:55 PM) omega8cc: from me - I made a few important changes to nginx configuration and plan to submit all of them shortly, also next week I plan to submit nginx installer, promised long time ago (03:11:59 PM) ergonlogic: nothing really on my end, though I've noticed some odd behaviour since updating to beta2, so I'll be looking through open issues and filing some bugs possibly this week (03:12:12 PM) anarcat: we need testers (dl new release and report bugs, test existing issues), documentors (community.aegirproject.org/handbook!) and developpers (submit and review patches and we will love you and give you commit access) (03:12:35 PM) anarcat: ergonlogic: awesome! (03:12:44 PM) anarcat: omega8cc: patches to hostmaster-install? or bash scripting? (03:13:28 PM) anarcat: otherwise, anybody else has somethign to say? (03:13:35 PM) ergonlogic: oh, also DNS for api.aegirproject.org should shortly be working (thanks Anarcat!), so check out that resource (03:13:42 PM) omega8cc: anarcat: mostly nginx conf templates, installer script (very simple bash) and some UI improvements (03:13:58 PM) ergonlogic: It exposes some pretty spotty documentation in the code (03:14:15 PM) anarcat: oh right, we are in the processing of transfering the dns to a new registrar (yay gandi.net!) and hopefully that will go without an hitch (03:14:23 PM) anarcat: omega8cc: i'd love to see this factored into hostmaster-install (03:14:33 PM) anarcat: ergonlogic: those could be reported as bugs, maybe... (03:14:43 PM) mig5: sure (03:14:45 PM) anarcat: ergonlogic: Vertice expressed interest in working on those documentation issues (03:14:55 PM) anarcat: okay, so that's pretty much it, thanks for attending our daily scrum! (03:15:03 PM) mig5: cheers (03:15:05 PM) anarcat: have a nice day, wherever you are! (03:15:09 PM) ergonlogic: that's what I had in mind, with maybe some doc patches where I can grok the code (03:15:16 PM) anarcat: can somebody file this in the OA site? (03:15:19 PM) darthsteven: Bad docs are bugs for sure (03:15:21 PM) omega8cc: anarcat: that is simple installer and how-to for standalone nginx based installs, (03:15:23 PM) ergonlogic: I got it (03:15:29 PM) anarcat: ergonlogic: thanks (03:15:37 PM) anarcat: scrum over, see you again next week :) (03:15:38 PM) anarcat: ciao ! (03:15:40 PM) darthsteven: Thanks everyone (03:15:43 PM) ergonlogic: Take care (03:15:47 PM) omega8cc: ciao
Weekly Scrum IRC Log: 2011-01-27
07:01 <@anarcat> morning! 07:01 <@anarcat> scrum is late omg! ;) 07:01 <@mig5> hai! 07:01 <@mig5> oops 07:01 <@anarcat> sorry i'm late :) 07:01 <@mig5> too many emails to read 07:01 <@anarcat> first rule of scrums : start on time damnit! ;) 07:01 < eft_> people still use email? 07:02 <@mig5> only 1 minute off. i blame timezones 07:02 <@anarcat> so anyways 07:02 <@anarcat> i almost caught up on all my issues 07:03 <@anarcat> i reported a few following significant upgrades 07:03 <@anarcat> maybe that was last week 07:03 <@mig5> I think so :) 07:03 <@anarcat> i want to make rc tonight 07:03 <@mig5> ! 07:03 <@anarcat> oh, we have deployed our first d7 platform here 07:03 <@mig5> nicely done.hefring in here runs on d7 07:04 <@anarcat> still waiting on actual tests for this - i think i'll just merge it in: http://drupal.org/node/1033072 07:04 <@anarcat> i agree with http://drupal.org/node/1039168 that path shouldn't be editable 07:04 <@anarcat> and well, that's it! 07:04 <@mig5> nice! 07:04 <@anarcat> i think that koumbit will rework its own roadmap next week, so i'll keep you up to date on that 07:04 <@anarcat> so how about an rc? 07:04 <@mig5> yes please do merge that in, i have heard a number of people add those indexes themselves and report good results 07:05 <@mig5> an rc sounds good 07:05 <@mig5> i would like to fix this http://drupal.org/node/1004526 07:05 <@mig5> this functionality got ripped out for some reason in like, June 07:05 <@mig5> so it's not critical, but i hate seeing good work disappear 07:05 <@anarcat> ah 07:05 <@anarcat> yeah, agreed 07:06 <@mig5> i'll briefly run through what i've been doing in the last 48 hours 07:06 <@mig5> i've mainly been trying to stabilise a bunch of shit 07:06 <@mig5> You could create platforms and servers with the same names/hostnames, leading to Duplicate entries and context clashes 07:06 <@mig5> http://drupal.org/node/1039010 http://git.aegirproject.org/?p=hostmaster.git;a=commitdiff;h=772cd8c2b5e... 07:07 <@mig5> Drush 4.x is broken in terms of rsyncing stuff to remote servers, so I reverted HEAD to use 3.3. see http://drupal.org/node/1041386 07:07 <@mig5> Drush Make beta10 is broken re: remote makefile referencing. So upped to beta11. http://drupal.org/node/979656 07:07 <@mig5> committed that fix for that Quota bug: http://drupal.org/node/1003666 07:07 <@mig5> Some deprecated ereg_replace thing in provision 07:07 <@mig5> uid 1 was not mapped to client 1 in hosting_client_user which broke some client-based platform access control http://drupal.org/node/996578 07:08 <@mig5> unconed identified an infinite loop with provision-save and contexts: http://community.aegirproject.org/node/267 07:08 <@mig5> I couldn't reproduce the infinite loop but can confirm clusters simply don't work 07:08 <@anarcat> about that uid1 thing - i thought uid1 was bypassing all access checks? 07:08 <@anarcat> i think ergonlogic had problems with that in the past too 07:08 <@anarcat> i see 07:08 <@mig5> it *was* at some point 07:08 <@anarcat> it'd be nice that node/267 be reported as a real bug... 07:08 <@mig5> but i decided, if we are letting people check a box granting access to that user, then maybe it should just work as expected 07:08 <@anarcat> too bad ppl report issues on community.a.o 07:08 <@anarcat> agreed 07:08 <@mig5> if it bypasses the checks, it should probably not even be a checklist option, 07:08 <@mig5> so i just fixed it the other way 07:08 * anarcat nods 07:08 <@anarcat> ok 07:09 <@mig5> yes unconed, He supplied me with a patch that seems to fix clusters! but concerned that I still couldn't reproduce the infinite loop. 07:09 <@mig5> xk has similar issues, so I put the patch there for review: http://drupal.org/node/1016890 07:09 <@anarcat> cool 07:09 <@mig5> and finally that Aliases thing 07:09 <@mig5> (phew! yes i did take notes in preparation for this scrum :) 07:09 <@mig5> that's me done 07:09 <@anarcat> man you're awesome 07:09 <@anarcat> do you plan to be as awesome next week? ;) 07:10 <@mig5> i can try, but i might balance it out by throwing in some migressions 07:10 <@mig5> :) 07:10 < eft_> my efforts are barely worth mentioning 07:10 < eft_> I didn't mention last week that I contributed a small wiki : using drush_make to optimize workflow http://community.aegirproject.org/node/256 07:11 <@mig5> eft_: yes, nice article! 07:11 <@anarcat> mig5: excellent :) 07:11 <@anarcat> okay, anyone else has anything to add? 07:11 <@anarcat> anybody scared of an RC? 07:11 <+omega8cc> This week (well, in the last minute) I rewrote and finally submitted for review two commits: one to add Nginx related how-to in the docs/INSTALL.txt - http://drupal.org/node/1042402 and second (too big probably, sorry!) with many improvements for Nginx configuration templates: http://drupal.org/node/1042312. I promise to submit future changes in the small incremental chunks instead of one big update. Next week I plan to submit even more Nginx related improv 07:11 <+omega8cc> and will debug mysterious issue - Aegir fails to rewrite paths in the "files" table on second and next site rename tasks, it works only on the first rename and I reproduced it on more than 5 imported sites, so something for sure interesting to debug and submit a patch (I hope). Also, I plan to finally work on some multiserver deployments and hope to help to find a fix for Drush 4.1 - http://drupal.org/node/1041386. That is it from me. 07:12 <@anarcat> omega8cc: your first line was stripped 07:12 <@anarcat> at Nginx related improv 07:12 <@mig5> nice, yes i freaked out the other day not knowing how to set up aegir with just nginx :) so those docs would be great 07:12 <@anarcat> thanks omega8cc ! :) 07:13 <@anarcat> your work is really appreciated, i wonder if we shouldn't just commit the huge patch with the hope that next ones will be incremental :) 07:13 <@anarcat> esp. considering you also use git, you should really "commit often" :) 07:13 <@anarcat> okay, anybody else? 07:13 <@mig5> anRC would be great - either way, yes definitely a new release, there are some annoying and critical bugs in beta2 (the data loss multiserver one for one) 07:13 < eft_> any word from devseed on moving c.a.o. ? 07:13 <@mig5> oh yes. or a.o 07:14 <@mig5> does anyone want to work on a drupalised version of a.o? and maybe give it to anarcat to host (or i can host it) 07:14 <@mig5> i can't theme for shit, so I'm out :) 07:14 < eft_> how much is there? 07:14 <@anarcat> mig5: i don't think we should drupalize a.o, actually 07:14 <@mig5> ah, cool that's fine with me 07:14 <@anarcat> just leave it like this, but remove the time-changing stuff 07:14 <@anarcat> and yeah we can host it 07:14 <@mig5> i remember adrian mentioning it 07:15 <@anarcat> oh, the DNS transfer of a.o and .com is stuck 07:15 <@mig5> everyone is annoyed at the g.d.o link, so if we could change that at least 07:15 <@mig5> ah 07:15 <@anarcat> email problems between gandi and devseed 07:15 <@anarcat> yeah 07:15 <@anarcat> but they wanted to transfer ownership of the domains too 07:15 <@anarcat> and i started the xfer 07:15 <@mig5> oh i see 07:16 <@anarcat> but anyways, yeah, transfer the NS and we'll go ahead from there 07:16 <@anarcat> okay, that's it for the scrum folks, thanks for attendign! 07:16 * anarcat heading for a sandwich 07:16 <@mig5> cheers 07:16 < eft_> later
Weekly Scrum IRC Log: 2011-02-03
06:59 <@anarcat> scrum time! 06:59 <@anarcat> mig5: you there? 06:59 <@mig5> yep 06:59 <@anarcat> omega8cc / ergonlogic / others... scrum starts now 06:59 <@anarcat> mig5: you wanna start? 06:59 <@anarcat> need to remove my skis 07:00 <@mig5> ok 07:00 <@mig5> i haven't done much, i still would like testers of http://drupal.org/node/1004526 07:00 <@mig5> but it sounds like we have two different issues there 07:00 <@mig5> and that i misunderstood the first issue 07:01 <@mig5> drush 4 remains unusable at least for multiserver, i had some difficulty convincing the drush guys that it was their problem http://drupal.org/node/1041386 07:01 <@anarcat> i wanted to test that and http://drupal.org/node/1016890 for ya before the RC 07:01 <@mig5> yes it's that cluster thing and the aliases that would be nice to get in before the RC 07:01 <@mig5> oh 07:02 <@mig5> and the package indexes 07:02 <@mig5> which i confirmed did seem to speed things up even on my tiny system 07:02 <@mig5> so do go ahead and merge that 07:02 <@mig5> that's all from me I think :s 07:02 <@anarcat> okay 07:02 <@anarcat> i can take a look at the drush 4 issue 07:03 <@anarcat> they are correct that the API is not supposed to change in dr4 07:03 <@anarcat> so if it's a dr bug, we'll have to wait until dr5 for a fix 07:03 <@mig5> yeah, on the other hand, they have completely reversed the logic 07:03 <@mig5> mmm. 07:03 <@anarcat> which is fine because dr5 is coming soon 07:03 <@mig5> o rly. fair enough! 07:03 <@anarcat> i'll weigh in on that one and we'll see 07:03 <@anarcat> on my side 07:03 <@anarcat> i haven't done much. :) 07:03 <@anarcat> still swamped with internal work here 07:04 <@anarcat> i wanted to do an RC, but failed to find time, i have scheduled time to review patches and do the RC on monday 07:04 <@mig5> oh nice, thanks 07:04 <@anarcat> maybe we should stick with 3.3 for aegir 4 07:04 <@anarcat> if you want to take care of the release before that, that's fine with me too though :) 07:04 <@anarcat> otherwise well i opened a few crazy issues 07:05 <@mig5> oh i need to try and reproduce omega8cc's report here: http://drupal.org/node/1047922 . not sure that's drush 4's fault, unless we know it works fine on 3.3 too 07:05 <@anarcat> most notably the exportable backup and wordpress support craziness 07:05 <@mig5> haha 07:06 <@anarcat> also, we have migrated to koumbit's DNS servers so i can do DNS changes if people need that to happen from now on 07:06 <@anarcat> api.aegirproject.org is up 07:06 <@anarcat> and aegirproject.org has been fixed 07:06 <@mig5> awesome stuff 07:07 <@anarcat> i think this is it for me too, i'm getting more and more towards freeing up my time to work on aegir, and we are aiming to work on our internal roadmap 'real soon' here at koumbit 07:07 <@anarcat> anybody else wants to cue in? 07:08 <@anarcat> anybody else is awake? :) 07:09 <@anarcat> so i guess this just makes for a short scrum! :) 07:09 <@mig5> no problem - i can go back to bed :) 07:10 < kvanderw> anarcat: what is a scrum? and could dummies help? 07:10 <@anarcat> thank you all 07:11 <@anarcat> kvanderw: http://community.aegirproject.org/scrums
Weekly Scrum IRC Log: 2011-02-10
07:00 <@anarcat> aaaalright 07:00 <@anarcat> hellow everybodyyy! 07:00 <@anarcat> welcome to our weekly scrums 07:00 <@anarcat> (we need a bot for that cheering crap - anyways) 07:00 <@anarcat> i 07:00 <@anarcat> have done nothing again. :P 07:00 <@anarcat> actually, that's not true 07:00 <@anarcat> i have done a good review of the queue 07:00 <@anarcat> to try to prepare for the RC 07:01 <@anarcat> committed a bunch of things lying around 07:01 <@mig5> you've done great 07:01 <@anarcat> and then univate decided to submit a bunch of patches on top of that, so i'll have to start all over again 07:01 <@anarcat> univate: thanks ;) 07:01 <@anarcat> yeah, it's been good 07:01 <@anarcat> i still have to review my own internal aegir shit before we go towards a release 07:01 <@anarcat> we have around 200+ issues in our internal queue, it's a mess 07:02 <@anarcat> we had a marketing meeting today where we determined our priorities, but i am not sure i should talk about that now ;) 07:02 <@mig5> :) 07:02 <@anarcat> expect another announce soon 07:02 <@anarcat> we are in talks with devseed to migrate the OA site (community.a.o) to Koumbit (? or mig5 ?) THIS MONDAY! 07:03 <@anarcat> so consider this the first announcement: 07:03 <@anarcat> ON MONDAY, 16H EST, the community site will go down for a migration 07:03 <@anarcat> (that's about the same time as now +1h) 07:03 <@anarcat> i still hope that we can RC next week, which will mean creating the 0.4.x branch 07:04 <@anarcat> but before that, i want to do some tests on my side and review our queue 07:04 <@anarcat> we had a weird issue with cron not running on a site here, that i have trouble making a diagnostic of 07:04 <@anarcat> with that 07:04 <@anarcat> i think i am done, mig5 ? 07:04 <@mig5> ok 07:04 <@mig5> not much to report, other than i think the only real blocker to getting an rc out, is that the upgrade path is broken 07:05 <@mig5> http://drupal.org/node/1056864 07:05 <@anarcat> i was scared to read that issue 07:05 <@mig5> i really need to speak to unconed about this since it's his patch, 07:05 <@mig5> and only he understands it, it's like talking to another Vertice :) 07:06 <@anarcat> good good :) 07:06 <@mig5> that's really all from me :s 07:06 <@anarcat> plans for next week? 07:06 <@mig5> to try and fix that, I suppose 07:06 <@anarcat> cool 07:07 <@anarcat> (as an aside, we just found this code in /var/aegir/fuck.php on our prod server, before destroying it, i pastebin'd it in the idea it might be useful for others: http://pastebin.com/163byC1z) 07:07 <@anarcat> (i am not sure what it does) 07:07 <@mig5> wtf 07:07 <@anarcat> okay, anyone else? 07:07 <+omega8cc> Last week I worked on some new features in the Nginx for Aegir configuration, like improved caching and performance (patches to be submitted), and tested everything with Drush 4 and 5 (head) 07:07 <+omega8cc> Now still working on issues like http://drupal.org/node/1056864, and hope to fix more Drush related issues next week to get Aegir Drush 4 compatible. That is it from me :) 07:08 <@anarcat> adrinux / darthsteven / EclipseGc / ergonlogic / grugnog_ / mvc / sfyn / skwashd / univate : you're all aegir contributors and welcome to say hi during this scrum - what you're working on these days... etc 07:08 <@anarcat> thanks omega8cc ! 07:08 <+darthsteven> Sure... 07:08 <@mig5> oh yeas darthsteven wrote some killer docs :) 07:08 <+darthsteven> I added the first part if some docs about context 07:09 <+darthsteven> More to follow, and extra docs about services too 07:09 <+EclipseGc> anarcat: oh awesome 07:09 <+darthsteven> That's it from me 07:09 <@anarcat> thanks darthsteven 07:10 <+EclipseGc> all of my time these days has been devoted to non-aegir-specific Drupal 7 work these days, though I do see a number of potential point of collaboration on various items in D7 for whenever aegir decides to make that jump 07:10 <@anarcat> EclipseGc: have you done d6 to d7 porting work? 07:10 <@anarcat> would you be interested in contributing to that? 07:11 <+EclipseGc> anarcat: I've worked on ctools/page_manager porting to D7 yes, as well as my own context_admin 07:11 <@anarcat> how's that going? :) 07:12 <+EclipseGc> anarcat: very good all things considered, native support for all entity types in page_manager is a pretty freaking awesome thing 07:12 <@anarcat> :) 07:12 <@anarcat> alright 07:12 <@anarcat> thank you EclipseGc 07:12 <@anarcat> anyone else has anything to add to this mighty scrum? 07:13 <+EclipseGc> anarcat: from an aegir perspective, (as an example) if say, "site nodes" became "site entities" in the D7 version, we could still utilize panels to display the site information (not that we're doing that now, but it is a possibility) 07:13 <@anarcat> oh, an idea i had: we should have a "wildcard domains" variable in the frontend for wildcards pointing to the aegir server while we fix DNS, then when people create sites, they can just type the left part of the domain and choose the wildcard from a dropdown 07:13 <@anarcat> EclipseGc: right, ok 07:13 <@anarcat> EclipseGc: should we wait for d7 before creating the "dns zone" content-type for example? 07:13 <@anarcat> how easy is it to declare entities programmatically? 07:14 <@anarcat> anyways, i have to head out now, so thanks everyone for attending, and have a nice day 07:14 <+EclipseGc> anarcat: well, the only way to declare an entity is programatically, there's no... entity ui or anything that allows you to build new entities 07:14 <@anarcat> can anyone take care of uploading this to the community site? 07:14 <@anarcat> EclipseGc: ok 07:14 <+EclipseGc> anarcat: thus far, a new entity, front to back all manual is minimum about 400 lines of code if you want it to be fairly usable 07:15 <@mig5> yeah i'll sort it later 07:15 <@anarcat> EclipseGc: ow? 07:15 <+EclipseGc> anarcat: I'm hoping that fago's entity_api and ctools can have a meeting of the minds and do something really great to make all of that (including user interfaces) much easier 07:16 <+EclipseGc> but that is still non-existent yet, and many people are uncomfortable depending on entity_api at the moment (though I hear it cuts down on the necessary code for an entity significantly) 07:16 <@anarcat> aand how easy is it to port existing content types to that ... stuff? 07:16 <+EclipseGc> anarcat: I'm going to change your vocabulary real quick (if you don't mind) just so there's no confusion 07:17 <@anarcat> i really have to head out, but feel free, i'll read on later tonight 07:17 <@anarcat> ciao everyone 07:17 <+EclipseGc> anarcat: "content types" or what we call node types in D6 and below are actually "bundles" (generic term) of the node entity now. You can still call them whatever you like, but to be precise I usually say "node bundles". All entity types can have bundles 07:18 <@anarcat> fuck 07:18 <@anarcat> sorry, screwed up there 07:18 <@anarcat> EclipseGc: got it :) 07:18 <@anarcat> ciao for real 07:18 <+omega8cc> cia 07:18 <+omega8cc> o 07:18 <+omega8cc> :)
Weekly Scrum IRC Log: 2011-02-17
07:00 <@anarcat> so welcome to our weekly scrum 07:01 -!- JoshBenner_ [~quassel@pool-173-59-14-26.phlapa.fios.verizon.net] has joined #aegir 07:01 <@anarcat> so to start with - i had a major fuckup with the community site with a verify that went bad 07:01 <@anarcat> because the alias didn't have the profile set in 07:01 <@anarcat> i documented all this in here http://community.aegirproject.org/node/355 07:01 <@mig5> reading that now. why did oyu need to manually run provision-deploy rather than standard migrate? 07:02 <@anarcat> mig5: because the site wasn't in aegir at first 07:02 <@mig5> gotcha 07:02 <@anarcat> i was importing from devseed 07:02 <@mig5> oh ofcourse *slaps forehead* 07:02 <@anarcat> maybe i should clarify that.. but anyways 07:02 <@mig5> no i'm just a dick and just woke up :) 07:02 <@anarcat> other than that, this week i have struggled with the issue queue 07:02 <@mig5> +1... 07:03 <@anarcat> specificall with that eugene guy which apparently decided we were evil and decided to develop his own stuff privately and stop collaborating on the queue 07:03 <@anarcat> i wrote a response in the issue queue, but i wanted to share here that it pissed me off real bad to hear somebody just go away like this because we're not being nice enough 07:03 <@anarcat> when he's treating us with abuse and duplicate issues 07:04 <@anarcat> hopefully he will not come back to the queue but share his work with others so that we don't reinvent the wheel 07:04 <@anarcat> one of those duplicate issues is this: http://drupal.org/node/1058918 07:05 <@mig5> his hosting_folders feature was just wrong anyway 07:05 <@anarcat> apparently eugene had a contrib module to fix this, but because i marked his issue as duplicate of mine, he got angry and removed his code from github 07:05 <@anarcat> but anyways, we talked about this vhost customization thing, and people were suggesting that it should be in contrib, but i feel like i disagree - if it's safe, it could be in core couldn't it? 07:05 <@mig5> the subfolder stuff? 07:06 <@anarcat> no no 07:06 <@anarcat> http://drupal.org/node/1058918 07:06 <@anarcat> the custom vhost stuff 07:06 <@mig5> ah 07:06 <+omega8cc> in core, imo 07:06 <@anarcat> right 07:06 < eft_> anarcat: that's moving vhost config into front end? 07:07 -!- Artusamak is now known as Artusamak_afk 07:07 <@anarcat> so anyways, that's about it for me - i'll try to work more on the issue queue and make an RC during next week unless there are objections here 07:07 <@anarcat> eft_: no 07:07 <@anarcat> mig5: next! 07:07 < eft_> anarcat: that's great that you migrated c.a.o. - do we still need the 1700 EST downtime in the header of this channel 07:07 <@anarcat> eft_: no 07:07 -!- anarcat changed the topic of #aegir to: Aegir hosting system 0.4-beta2 released! http://community.aegirproject.org/0.4-beta2 | Issue queue: http://is.gd/edalY | read this before asking: http://community.aegirproject.org/bugs | scrums on thursdays 2000UTC 07:08 <@mig5> i've done nothing all week due to some hectic work and basically some of our users killing my love for the project. 07:08 <@mig5> i fixed a minor bug here or there 07:08 <@mig5> and will likewise attack the queue some more, i think we should rc soonor we'll just be forever trying to battle that 07:08 <@anarcat> yeah 07:08 <@anarcat> agreed 07:09 <@mig5> that's it from me 07:09 <@anarcat> seems like the only blocker is the ssl rollback, which we could roll into another rc 07:09 * eft_ wishes he had more to offer than nitpicks about channel headers 07:09 -!- Artusamak_afk is now known as Artusamak 07:09 <@anarcat> let's just RC the fucking thing 07:09 <@anarcat> mig5: if you feel like it - just do it, and ignore the issue queue ;) 07:09 < eft_> mig5: any progress on your workflow diags? 07:09 <@anarcat> eft_: you can fight with trolls in the issue queue and get them off our backs :) 07:09 <+omega8cc> go with RC - to avoid "one more thing" syndrome == release 3 years later ;) 07:09 <@anarcat> there's a lot of triage work that is needed in the queue, that mostly anyone could do 07:09 <@mig5> eft_: that was just for a client proposal, it wasn't very good, http://greenbeedigital.com.au/files/aegir_workflow.jpg 07:10 <@anarcat> mig5: anything else? 07:10 <@mig5> anarcat: nope, that's it, i might release today then if i get time 07:10 < eft_> anarcat: bon cop, bad cop 07:10 <@anarcat> eft_: ouaip :) 07:10 <@anarcat> Vertice / adrinux / darthsteven / grugnog / mvc / omega8cc / sfyn / ergonlogic / univate_: anybody else got something to add to our scrum? 07:10 <@anarcat> mig5: that would be just amazing :) 07:10 <+omega8cc> not much from me this week, I only made barracuda/octopus compatible with Debian 6 and Ubuntu 10.10 plus some new fixes for Nginx config 07:11 <@anarcat> okay, anyone else? 07:11 <+ergonlogic> nothing from me 07:11 <+omega8cc> hope to work on contrib module for solr cores automatic setup - integrated with skwashd work 07:11 <@anarcat> univate_: thanks for those patches, btw, keep em coming! 07:11 <@anarcat> omega8cc: that would be freakin awesome! 07:12 <@anarcat> omega8cc: we got config for secure multi-core setups if you're interested... in our wiki 07:12 <+omega8cc> yep 07:12 <@mig5> anarcat: would love a review of http://drupal.org/node/1005014 07:12 <@mig5> (re:univate) 07:12 <@anarcat> mig5: got it, maybe tomorrow 07:12 <@anarcat> mig5: but don't wait for this to rc 07:12 <@mig5> ok 07:12 <+omega8cc> anarcat: I will check that, thanks! 07:12 <@anarcat> okay, anyone else? 07:13 <+ergonlogic> now that community.aegirproject.org is hosted at Koumbit, I could help if there's any issues with the site itself 07:13 <+ergonlogic> just let me know 07:13 <@mig5> eft_: wanted thesession_timeout something or other to be raised 07:13 <@mig5> so that it remembers you're loged in 07:13 <@mig5> logged* bah. 07:13 <@anarcat> ergonlogic: yeah, i think mig5 had issues with some images you could help with 07:13 <@anarcat> mig5: ^? 07:14 <@mig5> which images? 07:14 <@anarcat> mig5: i don't recall 07:14 <@anarcat> istr there was some thumbnail problem... wasn't there? 07:14 <@mig5> oh yes 07:14 < eft_> yea 07:14 <@mig5> possibly an atrium bug or something weird 07:14 <@anarcat> yeah 07:14 <@mig5> http://community.aegirproject.org/node/76 07:14 <@anarcat> well now we have root, so let us know what we need to do :) 07:14 <@mig5> the href to that is the full size image, but it *always* loads the thumbnail 07:14 <@anarcat> ergonlogic can take care of it or i can help 07:14 <+ergonlogic> ok, I'll take a look 07:15 <@anarcat> okay, i gotta go 07:15 <@anarcat> and the scrum is over, thanks for attending, people! 07:15 <@anarcat> and have a nice day 07:15 <@mig5> cheers 07:15 <+ergonlogic> thanks all 07:15 < eft_> ciao 07:15 <+omega8cc> ciao
Weekly Scrum IRC Log: 2011-02-23
17:00:00 * anarcat **** WEEEKLY SCRUM TIME!!! ****
17:00:07 <@anarcat> alright, welcome everybody
17:00:16 <jcapelo> ok thx
17:00:34 <@anarcat> darthsteven: sfyn EclipseGc Vertice : our weekly scrum has started!
17:00:40 <@anarcat> it seems that our new developer didn't make it
17:00:44 <@mig5> bugger
17:00:45 <+EclipseGc> :-(
17:00:50 <@anarcat> or is stuck behind the netsplit...
17:01:03 <@mig5> oh true, he was here before
17:01:13 <@anarcat> anyways, let's start
17:01:22 <@anarcat> so i have worked on upgrading our server to rc1
17:01:25 <@anarcat> our prod server
17:01:32 <@anarcat> and people are yelling at me because everything is slower
17:01:38 <@anarcat> so i wanted to yell back at univate ;)
17:01:41 <@anarcat> but he's not here :)
17:01:47 <@anarcat> so i'll just yell in the void
17:01:50 <@anarcat> which is better
17:01:53 <@mig5> have you tried his patch
17:01:57 <@anarcat> anyways i'll test the patches he sent to see if that fixes it
17:02:04 <@anarcat> but in the meantime i had a more urgent matter to attend to
17:02:10 <@mig5> i find it odd it only affects upgrades (fresh installs are quire fast for me)
17:02:19 <@anarcat> the UPPGRADE PATH (wooooo...)
17:02:20 <@anarcat> http://drupal.org/node/1068280
17:02:32 <@anarcat> mig5: it is quite odd
17:02:59 <@anarcat> mig5: maybe it's because upgrades have all modules installed and certain tasks (e.g. migrate) take more time to build their forms
17:03:07 <@mig5> ah!
17:03:17 <@mig5> thanks. that was annoyingly confusing
17:03:26 <@anarcat> what was?
17:03:33 <@mig5> not understanding why
17:03:35 <@mig5> and now i do :)
17:03:37 <@anarcat> anyways, i do think it's the wrong approach to build all forms now, it's slow
17:03:41 ::: mattmcmanus [~mattmcman@207.103.158.84] has quit [Quit: That's it! I QUIT! I can't take it anymore!]
17:03:47 <@mig5> yep. we copy temp tables etc there for migrate
17:03:50 <@anarcat> so maybe we should have a metadata field instead
17:03:52 <@anarcat> yeah, fuck it
17:04:05 <@anarcat> so we'll use the cache as a workaround, but send univate back to do his homework there
17:04:11 <@mig5> agreed
17:04:15 <@anarcat> alright\
17:04:30 <@anarcat> regarding the upgrade path, i think we agreed me and mig5 that he would do some regression testing on master, is that right?
17:04:41 <@mig5> sure thing
17:04:56 <@anarcat> there are two commits (one in hostmaster, one in provision) that are only in head and need to be merged in 0.4 once they are tested without regression and fixing the upgrade path from 0.3
17:05:23 <@anarcat> the one in provision needs to be actually reverted in master when the merge occurs too, because it's a 0.4-specific upgrade path that we can drop from master
17:05:24 <@mig5> ok
17:05:49 <@anarcat> i'll try to work full day on aegir tomorrow to try to settle some issues here
17:06:17 <@anarcat> i think that's all for me, i'm happy to see a new dev join the project, and i hope we can welcome more!
17:06:33 <@mig5> anarcat++
17:06:55 <@mig5> ok, as for me, i've done nothing as usual unfortunately
17:07:04 <@anarcat> as usual, i do not believe you :P
17:07:10 <@mig5> no, this time i haven't
17:07:14 <@mig5> i've been scared of the queue
17:07:17 <@anarcat> so what haven't you done
17:07:37 <@mig5> i'll test the upgrade path, i haven't even really looked at the issue queue since Eugen has discovered it :)
17:07:55 * EclipseGc just sliced his finger open on a grape juice package...
17:07:56 <@anarcat> istr that you actually answered a few things in the queue and here
17:08:08 <talengix> i have finally sucessfully installed aegir and i get Aegir is now installed. You can visit it at http://aegir.talengix.com/user/reset/1/1298497788/3644fc5f6ead4cf4d3a0c7... <talengix> but the page is blank. i have restarted httpd and I have ensured 127.0.0.1 localhost $AEGIR_HOST $AEGIR_DOMAIN are in my host file. what else could it be?
17:08:10 <@anarcat> EclipseGc: bleeding edge! ;)
17:08:15 <+EclipseGc> seriously
17:08:24 <@anarcat> alright
17:08:33 <@anarcat> anyone else has anything to add to our weekly scrum?
17:08:33 <@mig5> anyone else?
17:08:35 <@mig5> snap
17:08:36 <+EclipseGc> so, I set up a new rc1 for myself
17:08:37 <@anarcat> hehe
17:08:59 <+EclipseGc> trying to put together a hosted system for my own dev outside of the MAMP package
17:09:08 <+EclipseGc> so, first ++ really easy to install these days
17:09:26 <+EclipseGc> second, trying to hunt down why an install profile that works in MAMP won't install for aegir
17:09:57 <@anarcat> awesome
17:09:58 <+EclipseGc> kind of lost, but... I do have some clues, will need to follow up with people more knowledgeable than myself probably to get anywhere, but still... that's what I did
17:10:02 <@anarcat> ok
17:10:10 <@anarcat> are you still interested in the d7 port?
17:10:42 <+EclipseGc> I AM interested, I'm unsure of what I will have time for beyond advising
17:10:47 <@anarcat> i see
17:10:49 <@anarcat> ah mig5, regarding this: http://community.aegirproject.org/node/377#comment-411 - i think we'll have to live with some vagueness in the documentation, people will need to figure it out themselves, can't them?
17:11:01 <+EclipseGc> anarcat: sort of interested in trying to get ctools to a point to make that sort of dev easier for everyone
17:11:27 <@anarcat> mig5: if we have a thing that says something like "VERSION is the version you are trying to install, it's usually found on the frontpage of community.aegirproject.org" and use VERSION or OLD_VERSION everywhere in the
doc...
17:11:44 <@anarcat> EclipseGc: yeah, i was kind of scared of your ctools/features/hostmaster profile reshuffling there :)
17:11:50 <@mig5> anarcat: they ought to be able to, but sometimes it seems there's only maybe 10% of our users who have ever used linux before :) aegir is an 'attractive toy' it seems..
17:11:58 <@anarcat> EclipseGc: you should setup a sandbox for that on git.drupal.org when we migrate, so people can test it out
17:11:59 <@mig5> i am not that worried, i just want all of us to be aware of those issues
17:12:06 <@anarcat> mig5: awesome
17:12:07 ::: omega8cc [~omega8cc@85.17.159.47] has joined #aegir
17:12:07 ::: basic[~basic@iade105lmp02.blackmesh.com] has joined #aegir 17:12:07 ::: CitizenKane [~quassel@chat.qniformchat.com] has joined #aegir 17:12:08 ::: mvc [~mvc@shell.koumbit.net] has joined #aegir 17:12:08 ::: skwashd [~skwashd@phpgroupware/skwashd] has joined #aegir 17:12:08 ::: kvanderw [~kvanderw@mds-65-64-85-204.meridiandata.com] has joined #aegir 17:12:08 ::: gusaus [~gusaus@drupal.org/user/22137/view] has joined #aegir 17:12:08 ::: NeoID_ [~NeoID@cm-188.126.198.152.customer.telag.net] has joined #aegir 17:12:08 ::: TrevorT [~paulius.p@78.61.219.185] has joined #aegir 17:12:08 ::: SeanBannister [Sean@ppp118-208-132-239.lns20.bne1.internode.on.net] has joined #aegir 17:12:08 ::: josh_k [~josh_k@drupal.org/user/3313/view] has joined #aegir 17:12:08 ::: bertodsera [~quassel@109.251.3.19] has joined #aegir 17:12:08 ::: j0nathan [~j0nathan@modemcable012.160-130-66.mc.videotron.ca] has joined #aegir 17:12:08 ::: Chipie [~Chipie@mue-88-130-78-004.dsl.tropolys.de] has joined #aegir 17:12:08 ::: mthart [~mthart@pool-108-13-215-142.lsanca.fios.verizon.net] has joined #aegir 17:12:08 ::: joestewart|afk [~joestewar@69.172.231.35] has joined #aegir 17:12:08 ::: Zelfje [~Zelfje@30-80-ftth.onsneteindhoven.nl] has joined #aegir 17:12:08 ::: Letharion [~letharion@94-247-168-189-static.serverhotell.net] has joined #aegir 17:12:08 ::: univate [~chris@202-173-171-10.dyn.iinet.net.au] has joined #aegir 17:12:08 ::: ServerMode/#aegir [+vvvo omega8cc mvc skwashd univate] by leguin.freenode.net 17:12:08 ::: scyrma [~scyrma@drupal.org/user/1261/view] has joined #aegir 17:12:11 <@anarcat> frak 17:12:15 <@anarcat> okay 17:12:16 ::: basic
[~basic@iade105lmp02.blackmesh.com] has quit [Changing host]
17:12:16 ::: basic` [~basic@osuosl/staff/basic] has joined #aegir
17:12:18 ::: AquaticDisorder [~quassel@cpc11-chwo3-0-0-cust135.perr.cable.virginmedia.com] has quit [Ping timeout: 248 seconds]
17:12:18 <@mig5> whoa
17:12:23 <@anarcat> univate: you missed part of the scrum :)
17:12:25 <@anarcat> omega8cc: scrum time!
17:12:32 <@anarcat> mvc / skwashd : scrum time
17:12:51 <+EclipseGc> anarcat: yeah, I literally have 14 patches for ctools that I'm running in production
17:13:09 ::: Irishgringo [~chatzilla@c-66-229-63-143.hsd1.fl.comcast.net] has quit [Ping timeout: 272 seconds]
17:13:12 ::: catatonicrelapse [~catatonic@adsl-67-67-217-158.dsl.austtx.swbell.net] has joined #aegir
17:13:13 <@anarcat> summary of the scrum so far: i'm fixing the upgrade path, mig5 is scared of the queue and will do some regrsesion testing, EclipseGc has some patches for hostmaster.profile and is testing installs
17:13:14 <+EclipseGc> anarcat: so there's a lot of work left to do and a lot of signoff from earl left to get, but... I'm pretty confident
17:13:28 <@anarcat> we have this low-hanging fruit for documentors, which is reshuffling the installation manual
17:13:33 <@anarcat> http://community.aegirproject.org/node/377
17:13:41 <@anarcat> if somebody wants to start working on that, that would be awesome
17:13:59 <+EclipseGc> anarcat: also worth mentioning that, for REALLY big aegir installs, D7 hold some interesting search possibilities
17:14:11 <+EclipseGc> i.e. an aegir with lots of sites
17:14:14 <@anarcat> EclipseGc: ... and probably horrible performance issues possibilities ;)
17:14:29 <@anarcat> so anyways
17:14:30 <+EclipseGc> anarcat: well, not from search
17:14:34 <+EclipseGc> but D7 maybe yes...
17:14:36 <@anarcat> EclipseGc: right
17:14:45 <@anarcat> i think that's it for our scrum, which was a bit disturbed by our netsplit
17:14:59 <@anarcat> if people have things to add to the scrum, freel free to add that now
17:15:02 * EclipseGc shakes fist @ netsplit
17:15:06 <@anarcat> i don't have to run anywhere this time :)
17:16:33 <@anarcat> univate: we really need to rework that patch of yours for the VBO stuff, it's a release critical
17:19:31 <@anarcat> now for something completely different
17:20:11 <@anarcat> http://www.youtube.com/watch?v=CsQd2n99zS4
17:20:15 ::: scientist_ [~scientist@132.207.169.202] has joined #aegir
17:21:03 <+EclipseGc> anarcat: nice
17:21:10 <+EclipseGc> anarcat: definitely awesome
17:21:14 <@anarcat> :)
17:21:22 <@anarcat> i can take care of uploading the log into the community site for once
17:21:28 <@anarcat> unless mig5 has alraedy done it?
17:21:49 ::: Slydder1 [~chuck@dslb-088-072-212-004.pools.arcor-ip.net] has quit [Quit: Leaving.]
17:22:44 ::: joestewart [~joestewar@69.172.231.35] has joined #aegir
17:22:55 ::: bertodsera_ [~quassel@109.251.3.19] has joined #aegir
17:22:55 <+EclipseGc> anarcat: http://www.youtube.com/watch?v=dmoDLyiQYKw
17:23:15 <@anarcat> uh, and unless that wasn't clear, the scrum is over :)
Weekly Scrum IRC Log: 2011-03-09
09:01 <@anarcat> mig5: hey 09:01 <@mig5> up for scrum or inconvenient time? 09:01 <@anarcat> i can't believe i can't find those revisionsw http://drupal.org/node/1068280#comment-4131534 09:01 <@anarcat> hey, scrum time, look at that 09:01 <@anarcat> sure 09:01 <@anarcat> at a session here, but it still works :) 09:01 <@mig5> alright, go! :) 09:02 <@mig5> actually 09:02 <@mig5> first, i believe i saw you mention that this scrum time is inconvenient for you 09:02 <@anarcat> did something fuckup on d.o or something? 09:02 <@mig5> drupalcon aside 09:02 <@mig5> anarcat: you'll have to be more specific about d.o fuckups :) 09:02 <@anarcat> every first wednesday of the month i have a meeting during this scrum 09:02 <@mig5> ah. let's move it then 09:03 <@anarcat> what's convenient for ya? 09:03 <@mig5> the following day, but i think that's when it used to be for you 09:04 <@mig5> and you couldn't do later 09:04 <@anarcat> right 09:04 <@anarcat> that is correct 09:04 <@mig5> so maybe we go earlier in the week 09:04 <@anarcat> yeah, i can do the day before or before before 09:04 <@mig5> let's do day before, so Wednesday for us, unless it's a problem for univate 09:04 <@anarcat> got it 09:04 <@mig5> anyway! 09:04 <@anarcat> tuesday for us 09:04 <@mig5> d.o issues? 09:04 <@anarcat> right 09:05 <@anarcat> so about the upgrade path 09:05 <@anarcat> http://drupal.org/node/1068280#comment-4131534 09:05 <@mig5> i noticed i committed a change to provision and all my old commits are still unattributed to my user account. yay 09:05 <@anarcat> that comment refers to two patches that are supposedly on head 09:05 <@anarcat> and that i can't find 09:05 <@mig5> hmm 09:05 <@mig5> they're in master aren't they? 09:05 <@anarcat> i can't find them 09:05 <@anarcat> git fetch 09:06 <@anarcat> git show f64487706638 09:06 <@anarcat> fatal: ambiguous argument 'f64487706638': unknown revision or path not in the working tree. 09:06 <@mig5> http://drupalcode.org/project/hostmaster.git/commitdiff/75191804926c5c29... 09:06 <@mig5> http://drupalcode.org/project/hostmaster.git/commitdiff/a8dfee3391cb89f0... 09:06 <@mig5> are you sure your origin is not git.aegirproject.org ? :) 09:06 <@anarcat> um 09:07 <@anarcat> well 09:07 <@anarcat> those are not the commits i am looking for 09:07 <@mig5> oh 09:07 <@anarcat> that above comment refers to two other commits 09:07 <@mig5> oh oops 09:08 <@mig5> yeah i don't see it 09:08 <@mig5> weird 09:08 <@anarcat> oh well 09:08 <@anarcat> maybe i didn't push them and it works anyways... (?!) 09:08 <@anarcat> anywho 09:08 <@anarcat> so i wanted to merge shit in and release rc2 this week 09:09 <@anarcat> i talked to a bunch of people here at drupalcon about aegir 09:09 <@anarcat> i just did a aegir 101 session to about 20 people 09:09 <@mig5> nice 09:09 <@mig5> how'd that go 09:10 <@anarcat> pretty well 09:10 <@anarcat> people didn't really *want* to install aegir so that part didn't really work 09:10 <@anarcat> but we had good exchanges and dispelled common myths 09:10 <@anarcat> and gave away cool stickers :) 09:10 <@anarcat> we have a roadmap session planned for tomorrow 09:10 <@anarcat> at 2, in missouri again 09:10 <@anarcat> and then we are thinking of having a doc/code sprint on friday 09:11 <@mig5> nice 09:11 <@anarcat> maybe help people out writing drush extensions, and a bunch of other ideas 09:11 <@mig5> we have about 20 'for reviews' in the queues 09:11 <@anarcat> yeah 09:11 <@mig5> mostly eugen :P 09:11 <@anarcat> i'll try to throw people at those 09:11 <@anarcat> we may have new people to get into the project 09:11 <@mig5> excellent 09:11 <@anarcat> i met with shrop, EclipseGc and others 09:11 <@anarcat> what else 09:12 <@anarcat> i want to release rc2 :P 09:12 <@mig5> well - you can, i didn't see any errors doing an upgrade to master from 0.3 09:12 <@anarcat> i was hoping someone could shed light on this - but maybe i'll just do it myself: http://drupal.org/node/1081528 09:12 <@mig5> regardless of that missing commit 09:12 <@anarcat> oh, and i have pushed 6.x-1.x as the stable branch 09:12 <@anarcat> for provision 09:13 <@mig5> yeah that sounds expected: drupal keeps those paths in the db, you need to cache clear before it fixes itself up, so your theory is probably right there 09:13 <@mig5> that would happen with or without a Migrate task, basically 09:13 <@mig5> but would be nice for Migrate to fix it 09:13 <@anarcat> and for hostmaster now 09:13 <@anarcat> cool 09:14 <@anarcat> so i'll look at that before the release 09:14 < shrop> anarcat: you have aegir stickers? 09:14 < shrop> that would be cool 09:14 <@anarcat> shrop: yes! 09:14 <@anarcat> of course :) 09:14 < shrop> ok.. want one tomorrow at the meetup if you have any.. :) 09:14 <@anarcat> come to the bof tomorrow :) 09:14 <@anarcat> of course 09:14 < shrop> will do 09:14 <@anarcat> alright 09:14 <@anarcat> so what else for the scrum 09:14 <@anarcat> we did a little spike on the install docs manual for the purpose of the session 09:15 <@anarcat> i'd actually like to get a debian package working for rc2, but i'm not sure i'll have time to do this 09:15 <@anarcat> ideally, we'd get rc2 out during that code sprint 09:15 <@mig5> maybe for 1.0 :) 09:15 <@anarcat> ok, i think that's it for me 09:16 <@anarcat> well, i'd like the package to be tested :) 09:16 <@mig5> i've got nothing to report. univate, you here? 09:16 <@mig5> omega8cc? 09:16 < univate> hey 09:16 < hefring> eh oh 09:17 <@mig5> did you want to add anything to the scrum? 09:17 <@anarcat> mig5: i have merged the two commits you mentionned on the stable branch 09:18 <@mig5> cheers 09:19 <@anarcat> and then some more 09:19 <@mig5> i dont fully understand omega8cc's patches here http://drupal.org/node/949044#comment-4144944 09:19 <@mig5> + $form_state['values']['parameters']['new_uri'] = $url; // force lowercase 09:19 <@mig5> does that variable assignment really force lowercase? 09:20 <@mig5> maybe it's for readability. 09:20 <@mig5> anyway 09:21 <@anarcat> that doesn't force lowercase 09:21 <@anarcat> at all 09:22 <@anarcat> otherwise, i think that's it for our scrum 09:23 <@mig5> yep 09:23 <@mig5> thanks all
Weekly Scrum IRC Log: 2011-03-15
09:01 <@anarcat> alright 09:01 <@anarcat> scrum time 09:01 <@anarcat> mig5 / univate / Vertice / darthsteven / mvc / omega8cc : scrum time 09:01 <+omega8cc> hello! 09:01 < hefring> que tal 09:01 <@mig5> morning 09:01 <+darthsteven> Hello! 09:01 < hefring> hello 09:01 <@anarcat> ghankstef / mrbaileys / sfyn_ / shrop / / others i forgot: welcome to our weekly scrum! feel free to jump in! 09:02 <@anarcat> so welcome everybody 09:02 <@anarcat> we're back from drupalcon! it was an insane week 09:02 <@anarcat> so i'll start with that 09:02 <@anarcat> crazy 09:02 <@anarcat> week 09:02 <+omega8cc> and sk]washd! 09:02 <@anarcat> met a bunch of people 09:02 <@anarcat> oh yeah! skwashd ! 09:02 <@anarcat> even though we didn't have a formal aegir session schedule (which is strange considering there were 5 submitted), we did host 2 bofs and a training session, plus a code sprint 09:03 <@anarcat> i met with about 30-40 people to talk about aegir during those session 09:03 <@mig5> someone must've been anti-aegir on the organising committee :) 09:03 <@anarcat> for me the best session was the roadmap session, of which my coworker mvc posted a summary here: http://community.aegirproject.org/node/488 09:03 < skwashd> anarcat, we should've had 34 at the initimate session alone :) 09:03 <@anarcat> mig5: i actually suspect it's because there weren't "big names" submitting a session 09:04 <@anarcat> skwashd: that's right! 09:04 <@mig5> ah ;) 09:04 <@anarcat> and i was late to the intimate aegir session, which sucked, but it was my only "free" day during the whole week 09:04 <@anarcat> koumbit *drove* to chicago, which was insane 09:04 <@anarcat> 1400km one way, so around 3000 km of driving, 30h total... 09:04 <@anarcat> lots of fun and spilled coffee 09:04 <@anarcat> oh and i scratch the fucking car when parking it in a narrow street 09:05 <@anarcat> 4500$CAD in damage, believe it or not 09:05 <@anarcat> lots of fun 09:05 <@anarcat> anyways 09:05 <@mig5> haha 09:05 <@anarcat> so we're passing the hat to pay for the stupid koumbit van ... ;) (kdding) 09:05 <@anarcat> we *almost* got rc2 released 09:05 <@anarcat> i was about to push the tags when they cut off power and wifi in the code sprint 09:05 <@anarcat> but mig5 ended up doing the last mile on that 09:06 <@mig5> sort of 09:06 <@anarcat> i personnally met with shrop, cashwilliams, ghankstef, joestewart, and probably others i forget here :) 09:06 <@anarcat> i feel that our presence there got people interested in participating in the project and i got that objective at least partially accomplished 09:07 <@anarcat> with people comitting to watching and helping in the queues, and i helped people review patches in the code sprint 09:07 <@mig5> it sounded like it had a better bof/community spirit than san francisco 09:07 <@mig5> which is excellent 09:07 <@anarcat> yeah 09:07 <@anarcat> we did more BOFs 09:07 <@anarcat> i think it's better for us to do BOFs than formal sessions - people know of aegir, we need to make it work now :) 09:08 <@anarcat> i closed 10-20 tickets in the issue queue and pissed of eugen again 09:08 * mig5 claps 09:08 <@anarcat> we're down to around 5-10 needs review tickets 09:08 < ghankstef> it works great - except for when it doesn;t : ) but when it does its really really great 09:08 <@anarcat> this week i am working from home so i am hoping to get more aegir work done than usual 09:08 <@anarcat> but so far i'm failing, mostly because of internal poutine again 09:09 <@mig5> you've got chips and gravy inside you? 09:09 <@mig5> :) 09:09 <@anarcat> i hope to get into it tomorrow and get a solid 3 days 09:09 <@anarcat> haha 09:09 <@anarcat> i mean internal to koumbit :P 09:09 <@mig5> :) 09:09 <@anarcat> and it's french fries and gravy and cheese, for your information :) 09:09 <@anarcat> but enough with quebecois cuisine 09:09 <@anarcat> i think i'm done when i start to talk about poutine :) 09:09 <@anarcat> ah, yeah 09:10 <@anarcat> this week i hope to get working on "i want to run drush on my sites as a regular user" 09:10 <@anarcat> which promises to be a real PITA 09:10 <@anarcat> and also working on the 2.x roadmap, after feedback from the community 09:10 <@anarcat> so that's it for me - next! 09:11 <@mig5> i did nothing except push out rc2, which went horribly wrong, but mostly works 09:11 <@mig5> next! :) 09:11 <@anarcat> haha 09:11 <@anarcat> a bit more on that though - we weren't able to tag hostmaster, because of a bug on d.o 09:11 <@anarcat> we're pushing the limits on git and install profiles on d.o, and that's cool for those people, they like us :) 09:11 <@anarcat> everybody is quite happy and proud that aegir is back on d.o 09:11 <@anarcat> if feels good 09:11 <@anarcat> ah and for everyone that contributed to aegir here: 09:12 <@anarcat> i had a *lot* of thanks and good feedback on the project 09:12 <@anarcat> people *love* aegir 09:12 <@anarcat> they are very grateful and thank me for my involvement 09:12 <@anarcat> so i want to share that with the team here 09:12 <@anarcat> because i couldn't possibly have done this alone, so you all share this credit :) 09:12 <@anarcat> mig5: you really should have been there :) 09:13 <@mig5> i doubt i'll ever make another drupalcon 09:13 <@mig5> the two times of the year are awful for me frankly 09:13 <@mig5> anyways 09:13 <@anarcat> that's too bad 09:13 <@mig5> i'll look at that core dump/segfault issue you're having 09:13 <@mig5> i have never reproduced it 09:13 <@anarcat> you don't *have* to come twice a year :) 09:13 <@mig5> but i feel i understand the *reason* for unconed's patch 09:13 <@anarcat> if you have xdebug enabled, it's not a core dump, it's a loop 09:13 <@mig5> right 09:14 <@mig5> but i've never seen the segfault myself, is what i mean 09:14 <@anarcat> yeah 09:14 <@anarcat> you need to reference a non-existing platform 09:14 <@mig5> unconed explained it was something about the stdin context in drush that caused a loop of loading the @self or @server_msater alias 09:14 <@mig5> ahy 09:14 <@mig5> ah* 09:14 <@mig5> he tried to specifically exclude the @server_master context in his patch, which fixed clusters in this case, but broke install/migrate 09:14 <@mig5> anyway, that's for the tickets, i won't noise up the scrum now 09:15 <@anarcat> alright, anyone else? 09:15 <+omega8cc> I did nothing new "inside" the project last week. There are still some Nginx config improvements waiting for review in the queue, but.. 09:15 <+omega8cc> I released Barracuda and Octopus 1.0-boa-T to celebrate Aegir 1.0-rc2, which introduces support for Squeeze and Maverick, with even more speed improvements, so.. 09:16 <+omega8cc> I *think* we are already as fast as Mercury (formerly Pantheon) out of the box ;) Seriously, it needs some testing to confirm. That is it from me! 09:16 <@anarcat> alright! 09:16 <@anarcat> people looking for low-hanging fruits! 09:16 <@anarcat> **** THIS IS YOU! **** 09:16 <@anarcat> http://drupal.org/project/issues?text=&projects=provision,+hosting,+hostslave,+eldir,+Hostmaster+(Aegir)&status=8&priorities=All&categories=All 09:16 <@anarcat> this is the "needs review" queue 09:17 <@anarcat> what you need to do in there is to pick an issue in there, try to reproduce the issue described 09:17 <@anarcat> if you can't reproduce, close the issue (can't reproduce) 9:17 <@anarcat> if you can, try to apply the patch to see if it fixes the thing for you, if it does mark it RTBC (ready to be committed), if not, "needs work" 09:17 <@anarcat> there is also work to be done in the documentation 09:18 <@anarcat> we have reworked the manual, actually ergonlogic did 09:18 <@anarcat> http://community.aegirproject.org/notebook 09:19 <@anarcat> followup on documentation is in the "documentation" category in hostmaster 09:19 <@anarcat> see also http://community.aegirproject.org/node/377 09:19 <@anarcat> we specifically need help on the upgrade document 09:19 <@anarcat> and reorganising the "using aegir" section 09:19 <@anarcat> so if nobody has anything else to add 9:19 <@anarcat> i think that's it for our scrum this week 09:20 <@anarcat> thanks for everyone for attending 09:20 < ghankstef> I will have a look at the upgrade doc see if I can add to it 09:20 <@anarcat> ghankstef: alright! 09:20 < ghankstef> Will give me a good excuse to update an old aegir site 09:20 < ghankstef> oh yeah its all empty 09:20 <@anarcat> well 09:21 < ghankstef> guess I can improve on that a bit 09:21 <@anarcat> it's not really empty - i feel ergonlogic forgot to update the status page 09:21 <@anarcat> it seems the UPGRADE.txt was migrated into separate wiki pages 09:21 <@mig5> the sub pages are where it's at 09:21 <@anarcat> maybe it can be merged in one page 09:21 <@mig5> the wrapper page is a bit bare 09:21 <@anarcat> i think we should throw everything in the parent page there 09:22 <+omega8cc> and the structure is too deep now 09:22 <@anarcat> alright 09:22 <@anarcat> that's it for me folks, going back in my "trying to code" hole :) 09:23 <@mig5> thanks all 09:23 <+omega8cc> thanks!
Weekly Scrum IRC Log: 2011-03-29
08:58 <@anarcat> alright, so we're 1 minute away from the scrum, but i'll start anyways :P 08:59 <@anarcat> i'm a bit in a rush since we're going to replace a switch at the datacenter in about an hour 08:59 <@mig5> np 08:59 <@anarcat> which could affect community.a.o momentarily 08:59 <@anarcat> we had a few issues with the server 08:59 <@anarcat> it was replaced with a spare recently and we diagnosed a problem with a corrupted bios 08:59 <@anarcat> our hw provider is running more tests and we hope to get the original server back online shortly 09:00 <@anarcat> as for actual dev 09:00 <@anarcat> i have worked on provisionacl recently and got it in good shape 09:00 <@anarcat> i hope to deploy it in production here this week and get rid of our ugly sudo -u aegir kludges we were giving to workers 09:00 < Aurorus> sounds interesting anarcat 09:00 <@anarcat> so now not only can worker access modules/ files/ etc without problems, but they can run drush commands as their regular user 09:01 <@anarcat> *plus* this gives access to the regular drush aliases 09:01 <@anarcat> although there are still warnings (fixed in drush 5 and drush 4-head) 09:01 <@anarcat> i'm working on having site aliases respect drush -i 09:01 <@anarcat> i have worked a lot on the debian packages too, they are mostly ready 09:02 < Aurorus> anarcat: Are there plans to allow the deletion of sites and platforms that fail verify? 09:02 <@anarcat> provision is packaged, and there's a meta-package for hostmaster that downloads provision 09:02 <@anarcat> i also want to get the debian packages finished this week and we'll update the docs accordingly 09:02 <@anarcat> hopefully we can ship 1.0 with debian 09:02 <@anarcat> i started coordination of the rc3 release, but we have a significant blocker with d.o that refuses to release hostmaster 09:03 <@anarcat> see http://drupal.org/node/1105824 09:03 <@anarcat> for the drush issues: http://drupal.org/node/1104450 and http://drupal.org/node/957182 09:04 <@anarcat> i am going to work more on security in the coming weeks, i plan to work two weeks in a retreat in the woods with internet access :) 09:04 <@anarcat> i will make sure we don't bootstrap evil modules so that allowing access to modules and themes is secure 09:04 <@anarcat> i'd like to release rc3 tomorrow 09:05 <@anarcat> i am thinking of starting work on the d7 port and ldap integration for 2.x during the next two weeks 09:05 <@anarcat> and koumbit has done its internal roadmap and we'll be working hard on 2.x for the summer 09:05 <@anarcat> i think that's pretty much it for me! 09:05 <@mig5> nice one. you have given yourself a lot of work as usual :) 09:05 <@anarcat> i am glad to see that crazy stuff darthsteven has been doing here :) http://www.computerminds.co.uk/articles/aegir-tasks-daemon 09:05 <@anarcat> i think it should be merged in 09:06 <@anarcat> well, if anyone wants to pick up rc3 or similar things, be my guest :) 09:06 <@mig5> i read that: i think the queue thing would be good in aegir, hopefully doesn't have too many dependencies? 09:06 <@anarcat> drush especially needs a lot of love - and we'll need to work with them for aegir 2.0, which i suspect will support only drush 5 or above 09:06 <@anarcat> mig5: seems to be standalone contrib! 09:06 <@mig5> ok cool 09:06 <@anarcat> it's just a ghetto php daemon :) 09:06 <@anarcat> what Vertice didn't want :) 09:06 <@mig5> i thought it deopeneded on some obscure launchd-like OSX daemon 09:07 <@mig5> but i admit i didn't read it thoroughly 09:07 <@anarcat> darthsteven: recommends supervisor, but it's just to keep the php script alive - you could use whatever you want 09:07 <@anarcat> hey, we could even use the cronjob to restart the daemon if it fails 09:07 <@mig5> anyway: not much from me, as usual. i basically encountered some significant stability issues with the master branch, and fixed it as best as I could, though I worry i broke some of your ideas there. http://drupal.org/node/1106768 09:08 <@mig5> i found the bug while writing my aegir build test in jenkins, which was exactly what it was designed to do (although it didn't literally fail the test :) ) 09:08 <@mig5> and that's the other main thing i've been toying with (jenkins), although it's at an early stage in terms of aegir tests. 09:08 <@anarcat> cool :) 09:08 <@mig5> but i think we could use it more and more 09:08 <@anarcat> that is freakin awesome 09:08 <@anarcat> can you paste urls for that here? 09:08 <@anarcat> i'm really lazy, nevermind :) 09:09 <@mig5> sure http://jenkins.greenbeedigital.com.au:8080/job/aegir%20install/ 09:09 <@mig5> i'd like to at least give you or other developers access to that properly, so you can run tests 09:09 <@mig5> at my expense :) 09:09 <@mig5> as i figure you might catch migressions, and do me a favour 09:09 <@mig5> :) 09:10 <@anarcat> haha 09:10 <@anarcat> well, in this case the regression was mine wasn't it ;) 09:10 <@mig5> no matter 09:10 <@mig5> one other thing: 09:10 <@mig5> i'm less worried than you re: drupal.org's issues in releasing hostmaster 09:10 <@mig5> i think it would be nice 09:10 <@mig5> but it's not a blocker in my opinion 09:10 <@anarcat> ok 09:10 <@mig5> as our design works around it already 09:11 <@anarcat> ... since we depend on git 09:11 <@mig5> yeah 09:11 <@anarcat> ok 09:11 <@anarcat> but the release.sh doesn't know that :p 09:11 <@mig5> ah, true :) 09:11 <@anarcat> so i almost released a broken rc3 here :) 09:11 <@mig5> don't worry, i broke rc2 09:11 <@mig5> but i ignored it :) 09:11 <@anarcat> eh 09:11 <@anarcat> ok 09:12 <@anarcat> so if you want to break^Wrelease rc3 tomorrow, or anytime, in fact, be my guest 09:12 <@mig5> don't want to make more work for us in the release.sh though. but, i'm afraid i'm still crippled with a 'don't wait for drupal.org to catch up with us' mentality :) 09:12 <@mig5> ok 09:12 <@anarcat> that's alright 09:13 <@anarcat> alright, anyone else? 09:13 <@mig5> i'll have some time, this time tomorrow 09:13 <@anarcat> omega8cc doesn't seem to be here, so i'll talk for her a little :) 09:13 <@anarcat> she found a vhost injection vulnerability in the alias 09:13 <@mig5> oh yeah 09:13 <@anarcat> it was fixed in head and I *think* i merged in stable 09:13 <@mig5> did the security team say anything? 09:13 <@anarcat> which is why i wanted to make a release 09:13 <@anarcat> yeah 09:13 <@anarcat> no embargo, just release 09:14 <@anarcat> since it's not a stable release, it's ok 09:14 <@mig5> ok 09:14 <@anarcat> so don't tag it as a security release either, because then it gets unpublished and all the shit 09:14 <@mig5> she has some nginx batch updates in the queue as well, since we never really test those ourselves, we should probably just roll them in 09:15 <@mig5> they missed rc2 already 09:15 <@mig5> http://drupal.org/project/issues?text=&projects=hosting%2C+provision%2C+... 09:15 <@anarcat> yup 09:15 <@anarcat> a good review of the needs review patches would be good, but not mandatory 09:15 <@anarcat> too bad we got that silly security issue otherwise that would have been 1.0 :P 09:15 <@mig5> oh well. i admit i didn't read the security vulnerability properly. how was it exploitable? 09:15 <@anarcat> darthsteven: please do submit a patch for that stuff, it seems like gold 09:15 <@mig5> i spose i could find the ticket 09:16 <@anarcat> oh and i think we should fix this too: http://drupal.org/node/1108810 09:16 <@anarcat> Files in sites/example.org/private are accessible. 09:16 <@mig5> yeah 09:16 <@anarcat> this is the security issue: http://drupal.org/node/1098304 09:16 <@anarcat> i thought the private files was already done 09:16 <@mig5> i thought we could stick it in the platform vhost 09:16 <@mig5> the deny all 09:16 <@mig5> his .htaccess is probably being ignored for that reason 09:17 <@mig5> in that we don't have AllowOverride 09:17 <@anarcat> yes, the .htaccess are ignored, on purpose 09:17 <@mig5> yep 09:17 <@anarcat> okay folks, i need to go! 09:17 <@mig5> so i thought we could inject that into the template ourselves 09:17 <@mig5> sound sane? 09:17 <@mig5> ok 09:17 <@anarcat> yep 09:17 <@mig5> have fun with your switch 09:17 <@mig5> cheers 09:17 <@anarcat> hehe i will :) 09:17 <@anarcat> ciao ciao
Weekly Scrum IRC Log: 2011-04-05
(06:02:27 PM) anarcat: mig5 / Vertice / darthsteven / EclipseGc / mvc / omega8cc / sfyn / skwashd / ergonlogic : scrum time! (06:02:35 PM) EclipseGc: anarcat: yay (06:02:43 PM) anarcat: joestewart: scrum! (06:02:48 PM) darthsteven: crouch...touch...engage (06:02:52 PM) anarcat: shrop / univate : scrum! (06:02:57 PM) anarcat: alright (06:03:31 PM) anarcat: so welcome everybody, sorry i'm little late :) (06:03:35 PM) anarcat: i got tons of stuff :) (06:03:39 PM) anarcat: first (06:03:41 PM) anarcat: master is dead (06:03:52 PM) anarcat: being an anarchist and all, no masters no god and everything ;) (06:04:00 PM) anarcat: more seriously - we're all on 6.x-1.x now (06:04:09 PM) anarcat: it was too much of a pain to sync that all the time (06:04:20 PM) EclipseGc: heh (06:04:32 PM) darthsteven: anarcat: has the master branch actually been removed now? (06:04:40 PM) anarcat: darthsteven: yes, it's dead. (06:04:49 PM) darthsteven: cool (06:04:51 PM) anarcat: also, i have cleaned up the docs regarding the branch naming conventions here: http://community.aegirproject.org/node/187 (06:04:59 PM) ergonlogic: long live 6.x-1.x! (06:05:05 PM) anarcat: reviews of that from the docs team would be welcome (06:05:18 PM) anarcat: speaking of which, we talked about having maintainers for various sections of the project (06:05:28 PM) anarcat: the first of which is the docs team, spearheaded by darthsteven and ergonlogic (06:05:40 PM) anarcat: i don't remember the full list of teams, but i'll dig it out later (06:05:55 PM) anarcat: so congrats to the docs team for the awesome work so far, more on that from darthsteven ltr i guess (06:06:00 PM) anarcat: i did the roadmap for 2.0 (06:06:07 PM) anarcat: http://community.aegirproject.org/roadmap/2.0 (06:06:16 PM) anarcat: everyone is strongly encouraged to review and augment (06:06:21 PM) anarcat: and especially take tasks (06:06:48 PM) anarcat: koumbit will *not* do everything in there, but we *will* make a 2.0 release to deploy our changes in production, so the stuff that's not being taken care of will be pushed to 3.0 (06:07:12 PM) arianek is now known as arianekPHONE (06:07:13 PM) anarcat: i would love to get betas for 2.0 released by the end of summer, with rcs in automn (ie. koumbit runnin 2.x in production by the end of the summer) (06:07:21 PM) anarcat: we release rc3 (06:07:24 PM) anarcat: i want to release rc4 (06:07:26 PM) anarcat: tonight (06:07:32 PM) EclipseGc: yay (06:07:32 PM) anarcat: we have a bunch more changes (06:07:47 PM) anarcat: especially the symlink stuff (now sites are symlinked into /var/aegir/clients for easier access) (06:07:53 PM) anarcat: provisionacl has been updated to fix the perms there too (06:08:08 PM) anarcat: debian packages have been released and uploaded to debian.koumbit.net (06:08:11 PM) mig5: whoop, daylight savings screwed me. morning (06:08:23 PM) EclipseGc: morning mig5 (06:08:26 PM) anarcat: morning mig5 ! :) (06:08:47 PM) anarcat: there's been a few bugs with the debian packages, mostly with the new installs (because i'm mostly testing upgrades from our existing installs) (06:08:56 PM) anarcat: upgrades work well, installs need testing but should work now (06:09:11 PM) anarcat: the docs have been updated so that the manual install doesn't use install.sh (06:09:16 PM) anarcat: which has been removed from the tree (06:09:23 PM) anarcat: completely (06:09:33 PM) anarcat: and the manual docs have been updated to use hostmaster-install instead (06:09:40 PM) anarcat: and favor the debian automatic installs through packages (06:10:01 PM) anarcat: koumbit will deploy the stable branch in staging tonight or tomorrow and rc4 in prod tomorrow (06:10:06 PM) anarcat: that's about it for me i think (06:10:17 PM) anarcat: oh and i'm in a retreat in the woods to code like crazy for a week or two (06:10:31 PM) omega8cc: haha (06:10:31 PM) anarcat: i'm trying to work on the d7 port, but if people want to join that effort, that would be welcome (06:10:43 PM) anarcat: i'll also focus on bootstrap security, as i see fun (06:10:49 PM) anarcat: prioritisation welcome (06:10:53 PM) anarcat: ok, next! (06:10:58 PM) mig5: i've done nothing, i'm sick and also have busted my back. also, i fly to europe end of next week, what great timing (06:11:06 PM) mig5: so i don't expect you'll get much out of me until end of may now (06:11:31 PM) mig5: that's it for my pep talk (06:11:35 PM) anarcat: yay! ;) (06:11:38 PM) omega8cc: mig5: welcome to eu! (06:11:41 PM) anarcat: well, rest and get well :) (06:11:45 PM) mig5: thanks (06:11:57 PM) EclipseGc: I also have done nothing, but anarcat if you're doing a 7.x upgrade I would love to be available in any capacity that you might find useful. Be that review, or just as someone to bounce ideas at (06:12:08 PM) anarcat: EclipseGc: gotcha (06:12:19 PM) anarcat: so let's welcome our new dev here - darthsteven , do you have something to share? (06:12:33 PM) darthsteven: sure (06:13:02 PM) darthsteven: So, prompted by ergonlogic's issue, I asked for commit access to aegir to work on the documentation (06:13:13 PM) mode (+o darthsteven) by anarcat (06:13:14 PM) darthsteven: and it was granted (06:13:23 PM) anarcat: great timing :) (06:13:30 PM) EclipseGc: woot, now he can kick us all (06:13:36 PM) darthsteven: hurrah! (06:13:46 PM) darthsteven: (no idea how to though) (06:13:49 PM) anarcat: congratulations! :) (06:13:55 PM) darthsteven: so yeah, just been working on some docs (06:13:57 PM) anarcat: darthsteven: /kick EclipseGc now now (06:13:58 PM) anarcat: ;) (06:13:59 PM) omega8cc: darthsteven: congrats! (06:14:05 PM) EclipseGc: :-( (06:14:10 PM) omega8cc: lol (06:14:12 PM) darthsteven: and wrote an article about passing data from the Aegir frontend to the backend (06:14:12 PM) anarcat: EclipseGc: just kidding :) (06:14:15 PM) EclipseGc: heh (06:14:23 PM) ***EclipseGc saw that (06:14:32 PM) mode (+v shrop) by anarcat (06:14:45 PM) EclipseGc: http://www.computerminds.co.uk/articles/storing-data-aegir (06:14:58 PM) anarcat: darthsteven: that's nice! (06:15:04 PM) anarcat: darthsteven: we should copy that into the manual somewhere... (06:15:14 PM) darthsteven: from our work on the docs so far, I wonder if we wanted to clean up the API before the 1.0 final release? (06:15:23 PM) darthsteven: it's not hugely consistent (06:15:27 PM) EclipseGc: heh (06:15:38 PM) mode (+v ergonlogic) by anarcat (06:15:42 PM) darthsteven: but that would mean big changes I guess, so 2.0 stuff (06:15:54 PM) ergonlogic: I've been promoted (06:15:56 PM) anarcat: darthsteven: i would also think that api changes at this point would be a bit daring (06:16:14 PM) anarcat: darthsteven: i would rather tag those inconsistencies as issues for 2.x (06:16:17 PM) darthsteven: sure (06:16:19 PM) anarcat: darthsteven: but document, document (06:16:32 PM) anarcat: also, i think we should aim for a short 1.0 release cycle (06:16:35 PM) darthsteven: so, plans are to add lots more documentation over the next days/weeks (06:16:41 PM) anarcat: esp. because we're having api issues and so on (06:16:42 PM) darthsteven: that's it for me (06:16:46 PM) anarcat: alright (06:16:57 PM) anarcat: thanks darthsteven and welcome again in the crew (06:17:07 PM) anarcat: so a parenthesis on the statuses in this channel (06:17:13 PM) anarcat: +o is for developers (06:17:37 PM) anarcat: we're all equal there - if you need access to some resource on aegirproject.org or drupal.org, just ask and it will be granted (06:17:47 PM) anarcat: +v is for (currently) occasional contributors (06:17:53 PM) anarcat: it's the way to become a dev :) (06:18:09 PM) anarcat: end of parenthesis (06:18:19 PM) anarcat: so anyone else has something to add to our (late) scrum? (06:18:19 PM) anarcat: omega8cc: ? (06:18:53 PM) omega8cc: nothing from me this week, I'm sick, but hope to add something next week (06:18:57 PM) anarcat: i can speak quickly for sfyn, who made his first official release of uc_hosting (1.0-beta1), ubercart integration (06:19:06 PM) anarcat: which we're going to deploy soon (06:19:18 PM) anarcat: also, we'll need to figure out where the heck univate has gone :) (06:19:25 PM) anarcat: if anybody saw him, let me know ;) (06:19:49 PM) ergonlogic: So, for my part, I've actually been pretty busy... (06:19:57 PM) ergonlogic: testing some of the aegir contrib modules (06:20:09 PM) ergonlogic: some functional updates to the community site (spread out over the past few weeks really)... and more to come (06:20:19 PM) ergonlogic: Some documentation, though I feel I need to better understand how to document hooks properly... (06:20:28 PM) ergonlogic: I'll be trying to debug some issues on api.aegirproject.org (06:20:42 PM) ergonlogic: And I have a new Aegir contrib module (https://github.com/ergonlogic/hosting_saas) that denies uid1 to clients creating sites, and assignes them a role of our choosing (06:20:52 PM) ergonlogic: which I'll release shortly on d.o, I think (06:21:05 PM) ergonlogic: that's it, I think (06:21:06 PM) anarcat: for hooks, we need to add sample code for it to make sense, e.g. hook_post_install() (06:21:26 PM) darthsteven: ergonlogic: Feel free to ask me about documentation! (06:21:30 PM) EclipseGc: ergonlogic: nice (06:21:31 PM) anarcat: ah - ergonlogic and darthsteven - do we want to refactor your code straight into 1.0 core? or maybe we should wait for 2.x? (06:21:33 PM) ergonlogic: I've some experience with a coule now, so I can try to put something together (06:21:43 PM) ergonlogic: s/coule/could (06:22:14 PM) darthsteven: anarcat: we're working in a branch of 1.0 anarcat AntiNSA-AFK (06:22:32 PM) anarcat: darthsteven: for docs, but i was talking about the scheduling, the dispatcher, and saas stuff (06:22:46 PM) darthsteven: anarcat: ah okay (06:22:48 PM) anarcat: darthsteven: i think we can merge that branch in rc4 - it's really all just docs (06:23:10 PM) anarcat: i also think we can merge documentation into 1.x releases after 1.0 (06:23:23 PM) ergonlogic: I'd be happy for aegir core to appropriate hosting_saas (06:23:34 PM) anarcat: so anyways, i think that's it for our scrum (06:23:35 PM) ergonlogic: I think it might mean another promotion ;) (06:23:44 PM) anarcat: hopefully we'll have rc4 ready tonight or tomorrow (06:23:48 PM) EclipseGc: yay (06:23:53 PM) anarcat: and 1.0 ready by next week or something (06:23:54 PM) EclipseGc: mass migrate will come back :-D (06:24:00 PM) anarcat: EclipseGc: yeah, i fixed it (06:24:02 PM) darthsteven: cool (06:24:04 PM) EclipseGc: anarcat: yeah I saw (06:24:05 PM) ergonlogic: sounds good (06:24:10 PM) EclipseGc: anarcat: thx for that :-D (06:24:27 PM) anarcat: if someone wants to volunteer for the 1.x branch maintenance, i'd be happy to explain how that works and help with the work (06:24:42 PM) ergonlogic: (06:24:45 PM) EclipseGc: lol (06:24:48 PM) anarcat: ergonlogic: :P (06:25:02 PM) anarcat: ergonlogic / darthsteven : i think i was too quick about the offer to merge the modules in core - let's do that merge in 2.x instead (06:25:13 PM) darthsteven: anarcat: agreed (06:25:29 PM) anarcat: okay, who wants the amazing and wonderful job of uploading the log in the site? (06:25:31 PM) ergonlogic: k (06:25:38 PM) ergonlogic: I'll do it (06:25:47 PM) anarcat: great! (06:26:04 PM) anarcat: thanks everyone for coming
Weekly Scrum IRC Log: 2011-04-12
23:02 darthsteven: mig5: ping 23:02 darthsteven: anarcat: ping 23:02 darthsteven: Scrum? 23:02 hefring: Every Tuesday at 22h00 UTC: 15h00 San Francisco, USA (PST), 18h00 Montreal, Canada (EDT), 23h00 London, UK (CET), 08h00 (Wednesday) Melbourne, Australia (EST) 23:03 darthsteven: Guess this is going to be a quiet scrum then... 23:03 darthsteven: Well, I'll start 23:04 obicke: go for it 23:04 darthsteven: I've not done a whole lot of extra documentation, just all the constants in hostmaster, and a couple more hooks 23:04 darthsteven: I've been chasing ergonlogic about the api.a.o site not updating, he should be looking at it soon 23:05 darthsteven: hoping to still push some more docs before the 1.0 final release 23:05 darthsteven: that's it from me 23:05 darthsteven: anyone else? 23:06 darthsteven: EclipseGc, ergonlogic, grugnog, mvc, omega8cc, sfyn, shrop, skwashd or anyone else have anything to add? 23:06 obicke: • obicke is trying to round up anarcat and ergonlogic 23:07 EclipseGc: • EclipseGc was not reading 23:07 EclipseGc: • EclipseGc has nothing to add 23:08 shrop: Nothing at this point 23:08 shrop: When is v1 coming out? 23:09 darthsteven: soon 23:10 EclipseGc: • EclipseGc would like to point out his new power-module though http://drupal.org/project/path_profiler 23:11 darthsteven: I'll give this scrum until 22:15UTC 23:14 sethvincent: darthsteven: where are the api docs located? 23:15 darthsteven: sethvincent: api.aegirproject.org 23:15 sethvincent: thanks 23:16 darthsteven: that's the end of the scrum then
Weekly Scrum IRC Log: 2011-04-19
[22:00:51] <@anarcat> alright, welcome everybody to our weekly scrum [22:01:04] <@anarcat> i'm your host for this week and we have a wonderful agenda :) [22:01:09] <@anarcat> enough with the bull [22:01:14] <@anarcat> 1.0 IS RELEASED!!!!! [22:01:17] <@anarcat> YEEHAAA! :) [22:01:24] <@mvc> :) [22:01:26] <@darthsteven> YEAH!!! [22:01:26] <@anarcat> *AND* we have found our first RC bug! [22:01:33] <@anarcat> http://drupal.org/node/1132514 [22:01:38] <@anarcat> d5 migrations are fucked! :) [22:01:42] <@anarcat> so that's already fixed and great [22:01:48] <@anarcat> and i am thinking of making 1.1 already [22:02:06] <@anarcat> also, we found a low-hanging fruit here to make exportable backups easy: http://drupal.org/node/1047992 [22:02:17] <@anarcat> and i merged this in by mistake in 1.x, but now i think i won't revert it [22:02:36] <@anarcat> http://drupalcode.org/project/provision.git/patch/1edd757f95b3f86a84ba72... [22:02:43] <@anarcat> if anyone opposes, feel free to revert [22:03:03] <@anarcat> it's not an API change, and it's a nice quick fix to have people with SFTP access able to download backups of their sites [22:03:09] <@anarcat> so i think it would be nice to get this in [22:03:18] <@anarcat> koumbit is now running 1.0 in production [22:03:35] <@anarcat> and we're going to test head 1.x tomorrow, and if that works well we'll release 1.1 [22:03:46] <@anarcat> what else, what else [22:04:04] <@anarcat> did i mention we have a candidate student for the summer of code? [22:04:16] <@anarcat> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/sdv_3... [22:04:19] <@anarcat> here, i mentionned it [22:04:24] * mapleleaf has quit (Quit: ChatZilla 0.9.86.1 [Firefox 3.6.16/20110319135224]) [22:04:24] <@anarcat> results should be announced somewhere next week [22:04:33] <@anarcat> his focus would be on the d7 port and maybe dns [22:04:40] <@anarcat> so i decided to hold off on those tasks for now [22:04:54] * Chipie has quit (Quit: Leaving.) [22:04:55] <@anarcat> i will however look more at inter site security [22:04:57] <@darthsteven> all very cool [22:05:02] * FatGuyLaughing has quit (Quit: FatGuyLaughing) [22:05:09] <@anarcat> and queueing, and i'd like to talk with darthsteven about that, but prolly more next week [22:05:16] <@anarcat> so that's about it for me [22:05:49] <@anarcat> mig5 darthsteven ergonlogic mvc omega8cc sfyn shrop skwashd - something to add to our weekly scrum? [22:05:55] <@darthsteven> yup [22:06:20] <@anarcat> oh, and of course, i forgot: http://community.aegirproject.org/discuss/2x-branch-created 2.x is opened for dev, and i merged the chained cert support there [22:06:33] <@darthsteven> I've just come from a Drupal Meetup where I saw that drupal 5 migration bug demoed [22:06:42] <@darthsteven> and now I know what caused it [22:06:44] <@darthsteven> :) [22:06:58] <@anarcat> and now you can tell them it's fixed already :)_ [22:07:17] <@darthsteven> So I took a look at at patch to add SSL chained certificates to provision [22:07:32] <@darthsteven> It needed a bit of cleanup, but was okay [22:07:43] <@anarcat> i agree, and merged it into 2.x [22:08:06] <@darthsteven> I think I added some docs since the last scrum, and ergonlogic got the api.c.o site working [22:08:36] <@darthsteven> ermm...that's it for me, nothing else Aegir related really [22:08:45] <@darthsteven> hoping to write more docs though... [22:08:51] <@darthsteven> done [22:08:55] <@darthsteven> anyone else? [22:09:09] <@anarcat> cool [22:09:22] <@anarcat> omega8cc: ? [22:09:35] <@anarcat> omega8cc: you mentionned something about tracking 2.x, can you expand on that? [22:09:36] <@omega8cc> I released Barracuda/Octopus 1.0-boa-T-8 with Aegir 1.0 [22:10:08] <@omega8cc> anarcat: just switched to 2.x to continue with patches related to Nginx [22:10:22] <@anarcat> i am, in general, a bit worried that we won't get as much testing on 2.x than we have on 1.x now that 1.x is declared stable, but i'll try to run 2.x on my home server - we were also thinking of running 2.x on a dev server at koumbit, but that has yet to happen [22:10:26] <@anarcat> omega8cc: cool, nice to hear [22:11:04] <@omega8cc> anarcat: I will use 2.x in Barracuda head, so people will test it and I hope to get more feedback [22:11:08] <@anarcat> cool [22:11:31] <@anarcat> i was thinking of reworking the platform access control for april, we set that as a target, so maybe that will land in 2.x and maybe get merged in 1.x if it doesn't break any apis [22:11:41] <@omega8cc> interesting that 5.x to 6.x to 7.x migrations worked for me [22:11:48] <@anarcat> i am still looking for ways to radically change 2.x so that it's really different from 1.x :P [22:12:01] <@omega8cc> but I tested it only on basic core installs [22:12:07] <@anarcat> i see [22:12:12] <@anarcat> nice to hear anyways :) [22:12:13] <@ryanarmstrong> anarcat: cross-site stats :P [22:12:21] <@ryanarmstrong> *goes back into the background* [22:13:25] <@anarcat> oh, darthsteven - this could use some more eyes as it touches on the context/services stuff that you may be familiar with: http://drupal.org/node/1074966 [22:13:41] <@anarcat> omega8cc: thanks for your feedback [22:13:44] <@anarcat> anyone else? [22:13:54] <@anarcat> ryanarmstrong: you'll have to expand on that after the scrum :) [22:14:03] <@ryanarmstrong> will do, didn't want to derail :) [22:14:07] <@omega8cc> we need UI improved in 2.x, I will submit patches soon, and will propose Fetaures integration etc [22:14:17] <@anarcat> omega8cc: great idea! [22:14:31] <@anarcat> omega8cc: one idea about features was to drop our idea of "features" in favor of features.module [22:14:38] <@darthsteven> anarcat: yeah I saw that, and it is on my radar, but I've not used any of the DNS stuff, but could look at the code anyways [22:14:57] <@omega8cc> right, and move install profile to profiler [22:15:06] <@omega8cc> maybe [22:15:10] <@anarcat> omega8cc: yeah [22:15:20] <@anarcat> omega8cc: ergonlogic was saying it was a good upgrade path to d7 [22:15:33] <@anarcat> alright [22:15:35] <@omega8cc> anarcat: yes, noticed that [22:15:47] <@anarcat> we're pretty much done here, thanks for attending our scrum session and see you next week! [22:16:02] <@darthsteven> I'm not sure that converting to features features would get us much, as we don't use CCK/fields [22:16:07] <@anarcat> anyone wants to upload this log to the com site? [22:16:26] <@darthsteven> will do [22:16:28] <@anarcat> darthsteven: indeed.. [22:16:31] <@anarcat> darthsteven: cool! tx [22:16:59] <@anarcat> darthsteven: as for the dns issue, it's not as much for the DNS but for the part where he changes the way config information is stored in the services objects [22:17:03] <@anarcat> hairy stuff [22:17:09] <@anarcat> alright, scrum end!
Weekly Scrum IRC Log: 2011-04-26
[22:00:05] <anarcat> hello everyone!
[22:00:09] <EclipseGc> 'ello
[22:00:10] <anarcat> welcome to our weeeeekly scrum!
[22:00:27] <anarcat> tons of great news this week
[22:00:34] <anarcat> sethvincent: scrum time
[22:00:41] <sethvincent> hello!
[22:00:41] <hefring> hey
[22:01:05] <anarcat> so alright, i'll start by welcoming our new google summer of code student that will work on the d7 port and dns frontend
[22:01:12] <anarcat> sethvincent: congratulations on your admission! :)
[22:01:23] <sethvincent> thanks! i'm pretty excited about it
[22:01:24] <anarcat> now, for my stuff
[22:01:28] <EclipseGc> sethvincent: congrats
[22:01:56] <anarcat> i worked some time on the drush unit testing stuff and have a patch for drush 5 to refactor the unit testing code so we can test functions directly
[22:02:14] <anarcat> i will work on unit testing before reshuffling the backend too much, which should make 2.x more robust throughout the refactoring
[22:02:30] <anarcat> we upgraded everything to 1.1
[22:02:34] <anarcat> which i released last week
[22:02:46] <anarcat> and our stable release seems to be well.. pretty stable :)
[22:03:01] <EclipseGc> heh
[22:03:02] <anarcat> sethvincent: i was wondering if we could make an announcement for your project, that would make nice publicity :)
[22:03:14] <sethvincent> ok. that sounds good.
[22:03:30] <anarcat> oh, and i met with ubuntu people at the local launch party to get help with the backport of drush in there, should followup shortly
[22:03:33] <anarcat> that's it for me
[22:03:34] <shrop> hey hey
[22:04:20] <anarcat> oh, next week, i'll work on that announcement, we're deploying provision ACL internally big time here, and i'll work more on the new jenkins server we have (thanks mig5!) and unit testing - that's all
[22:04:25] <anarcat> anyone else?
[22:04:33] <anarcat> sethvincent: maybe you want to introduce yourself a little? :)
[22:04:45] <sethvincent> sure.
[22:05:19] <sethvincent> i'll be working on the d7 port and dns.
[22:05:32] <anarcat> alright :)
[22:05:41] <anarcat> and i'll be your mentor, from what i understand :)
[22:05:44] <sethvincent> i'm a student at the evergreen state college, where i also work part time as an advisor to the student newspaper there
[22:06:04] <anarcat> (if anyone else wants to help with that, that would be cool, btw)
[22:06:36] <shrop> d7 ports and dns stuff?
[22:06:37] <EclipseGc> is the plan to move all our major components to entities?
[22:06:47] <sethvincent> yeah, i'm excited to hear any advice y'all have on the d7 port and dns
[22:06:52] <anarcat> right
[22:06:55] <sethvincent> entities does look like the way to go
[22:06:59] <anarcat> so we alraedy have a few things on the table here
[22:07:10] <anarcat> univate has a patch to refactor the install profile with profiler and use features more
[22:07:16] <anarcat> that will certainly facilitate the port
[22:07:30] <anarcat> that's in http://drupal.org/node/517620
[22:07:45] <anarcat> as for entities, yes, i figured that most of our current content type should probably become entities
[22:08:11] <anarcat> that, in turn, will facilitate implementation of the DNS frontend, which will be also based on entities
[22:08:21] <anarcat> and will provide a good use case for creating a new "hosting" content-type
[22:08:36] <anarcat> having documentation on how to actually do that while it's coded would be useful too in the manual
[22:08:52] <anarcat> i think that's all my brain has to offer now for that :)
[22:09:01] * FatGuyLaughing has quit (Quit: FatGuyLaughing)
[22:09:08] <anarcat> sethvincent: you should get familiar with git and work on a development branch, at the very least
[22:09:26] * FatGuyLaughing has joined #aegir
[22:09:34] <shrop> anarcat: Will there be a dev branch for d7? sorry if I missed that
[22:09:37] <anarcat> or maybe work in a sandbox for the time being, not sure it's appropriate to give you full commit access from the start
[22:09:41] * FatGuyLaughing has quit (Client Quit)
[22:09:42] <anarcat> shrop: there will certainly be!
[22:09:53] * realityloop has joined #aegir
[22:09:59] <shrop> cool. Just checked and didn't see it yet, but wanted to be sure.
[22:10:04] <sethvincent> ok. i have some git experience. i'll start out with a sandbox
[22:10:32] <anarcat> cool
[22:10:37] <omega8cc> sethvincent: congrats!
[22:10:58] <anarcat> awesome
[22:11:09] <EclipseGc> sethvincent: be sure to let us all know where you sandbox is
[22:11:16] <EclipseGc> your*
[22:11:29] <sethvincent> ok. what will be the best way to share things like that?
[22:11:34] <anarcat> i'm sure it will be quite popular :)
[22:11:41] <anarcat> hum
[22:11:43] <shrop> Just caught up on sethvincent's position here.. great!
[22:12:11] <anarcat> sethvincent: maybe here? http://community.aegirproject.org/maintainers
[22:12:22] <sethvincent> anarcat: ok
[22:12:23] <anarcat> omega8cc: maybe you want to add pointers to your repos in there, btw
[22:12:43] <anarcat> sethvincent: otherwise, what's the next step for you and us? :)
[22:12:44] <omega8cc> anarcat: right, good idea
[22:12:59] <anarcat> and if anyone else has anything to add to our scrum, now is a good time - we're almost done!
[22:13:04] <anarcat> darthsteven, you awake? :)
[22:14:19] <sethvincent> anarcat: i need to study some of the general changes from d6 to d7, the aegir api, and write out a plan for the d7 port.
[22:15:13] <sethvincent> google code expects students to start coding somewhere around may 24, but i'd like to do a slow start earlier than that so that by may 24 i've already made some progress.
[22:15:33] <anarcat> i see
[22:15:43] * arianekWORK has quit (Quit: Smell ya later)
[22:15:44] <anarcat> so between now and the 24th you start documenting a plan is that right?
[22:16:13] <anarcat> the right place for this is probably the handbook on the community site
[22:16:19] <sethvincent> yeah. a plan, studying documentation, and getting to know the mentors is what's listed on the google summer of code site, i think.
[22:16:20] <sethvincent> ok
[22:16:51] <anarcat> i have done some of that documentation here: http://community.aegirproject.org/developing/architecture
[22:16:56] <anarcat> before starting some coding, at least
[22:17:01] <sethvincent> ok
[22:17:49] <anarcat> alright, so welcome on board again, and thanks everyone for attending our scrum session!
Weekly Scrum IRC Log: 2011-05-03
[22:01:15] * anarcat SCRUM TIME!
[22:01:24] * anarcat cues boom-boom beat
[22:01:30] <anarcat> alrighty
[22:01:40] <anarcat> so
[22:01:56] <anarcat> what i'll do this week: finish setting up the new jenkins server, and work on drush/provision unit testing in 2.x
[22:02:07] <anarcat> i have fixed a pesky issue in the site form editing
[22:02:09] <shrop> Haha
[22:02:22] * trainer has joined #aegir
[22:02:31] <anarcat> and we're struggling with a core dump in d7 installs on our production server we can't reproduce in staging, really weird
[22:02:39] <anarcat> if anyone got more info, let me know
[22:02:52] <anarcat> that's it for me
[22:03:02] <anarcat> anyone else?
[22:03:07] <sethvincent> i'll go
[22:03:11] <anarcat> c_mcintosh: you said you had something to mention for our scrum?
[22:03:18] * arianek is now known as arianek_afk
[22:03:20] <anarcat> sethvincent: go ahead!
[22:03:28] <sethvincent> sandbox: http://drupal.org/sandbox/seth.vincent/1140244
[22:03:43] <sethvincent> and the plan: http://community.aegirproject.org/roadmap/2.0/d7-port-dns-editor-plan-go...
[22:03:51] <gmclelland> is hostmaster-pause still a valid drush command? It tells me could not be found?
[22:04:01] <sethvincent> if anyone has advice on the plan that i wrote up please let me know
[22:04:08] <anarcat> gmclelland: please wait after the scrum :)
[22:04:20] <EclipseGc> sethvincent: I'll read through it
[22:04:33] <sethvincent> i've been doing studying of the docs, and that's what i'll keep working on over the next week
[22:04:47] * scottrigby has joined #aegir
[22:04:48] * anarcat reading up
[22:04:49] <sethvincent> @EclipseGc thanks!
[22:04:57] <anarcat> sethvincent: sounds good!
[22:05:04] * arianek_afk has quit (Remote host closed the connection)
[22:05:06] <sethvincent> cool. that's it for me.
[22:05:20] <anarcat> sethvincent: do you plan on reviewing the existing work in the issue queue?
[22:05:25] <anarcat> that's where most of the stuff is so far
[22:05:31] <sethvincent> ah. yes
[22:06:13] <anarcat> i'm happy to see a plan, so far it looks sound
[22:06:23] <sethvincent> ok
[22:06:53] <anarcat> sethvincent: you should authentify with nickserv when you hang around here, it will give you +v, which is a good way to get notified when the scrum starts
[22:07:03] <EclipseGc> I'm hoping to get to do this this week or next, but I'm actually intending on screwing around with drupal commerce some to see if I can get an outside of aegir D7 + DC platform requesting that aegir provision new sites
[22:07:19] <anarcat> we may make the channel +m (moderated, so only voiced users can talk) eventually so that will make it really important you authentify
[22:07:23] <sethvincent> anarcat: ok. i'll do that
[22:07:23] <anarcat> and that goes for everybody :)
[22:07:45] <anarcat> EclipseGc: cool - note that c_mcintosh is working on that
[22:08:05] <EclipseGc> anarcat: http://drupal.org/project/hostmaster_plus I assume?
[22:08:07] <anarcat> koumbit is also rolling out a new ecommerce frontend that will talk with aegir through XMLRPC, maybe in collaboration with c_mcintosh, we'll see
[22:08:42] <anarcat> wtf
[22:08:43] <anarcat> no
[22:08:57] <anarcat> c_mcintosh: that's not a good module name
[22:09:12] <anarcat> c_mcintosh: first, it should be called hosting_* as hostmaster is the profile name
[22:09:19] <anarcat> c_mcintosh: second, "plus" doesn't mean anything
[22:09:30] <EclipseGc> anarcat: I was just intending on trying to get a straight up drush provisioning call going and requiring that they be on the same server
[22:09:38] * Met4physica has quit (Quit: Leaving)
[22:10:31] <anarcat> EclipseGc: not sure what you mean there
[22:10:40] <EclipseGc> anarcat: meh, no worries
[22:11:04] <EclipseGc> anarcat: I've not even gotten to dig in yet, so, nothing to see here ;-)
[22:11:09] <darthsteven> my turn?
[22:11:36] <anarcat> darthsteven: go ahead! :)
[22:12:11] <darthsteven> Well, I looked at running cron using wget for D7 and popped a fix into the 2.x branch
[22:12:35] <darthsteven> I think the change is pretty simple, and could be easily backported to 1.x
[22:12:59] <darthsteven> Also, I have been thinking about the scheduled task problem
[22:13:24] <darthsteven> so plan for coming week is more docs and coming up with a firm plan for scheduled/recurring task
[22:13:29] <darthsteven> that's all
[22:14:15] <anarcat> cool
[22:14:21] <anarcat> i commented on your issue, but basically it's a great idea
[22:14:30] <anarcat> so we're almost done with our scrum here
[22:14:45] <anarcat> if anyone else has something to add, speak now or be silent for... a week ;)
[22:14:51] <anarcat> just kidding, you can speak whenever :P
[22:14:55] <omega8cc> I don't have anything special to report this week - it was a little spring holiday for me, but next week I plan to work on some Nginx related docs for c.a.o.
[22:14:57] <anarcat> but now is the best time to get feedback!
[22:15:02] <anarcat> awesome
[22:15:09] <anarcat> omega8cc: thanks for that cron.php fix btw
[22:15:13] <darthsteven> awesome module: http://drupal.org/project/hosting_platform_pathauto
[22:15:24] <anarcat> you guys were making me crazy with that to and fro :)
[22:15:42] <anarcat> darthsteven: yeah, that's freaking awesome
[22:15:46] <anarcat> we need to merge it into 2.x
[22:15:53] * ipwa has quit (Quit: ipwa)
[22:15:55] <darthsteven> or just add to the makefile?
[22:16:42] * sethvincent has quit (Quit: Page closed)
[22:17:31] <EclipseGc> add to makefile
[22:17:32] <EclipseGc> ftw
[22:17:46] * dob_ has quit (Read error: Connection reset by peer)
[22:17:50] <anarcat> i think it should be in
[22:18:07] * dob_ has joined #aegir
[22:18:15] <anarcat> but whatever...
[22:18:20] <darthsteven> anyone else
[22:18:24] <anarcat> the biggest problem right now is that it hardcodes /var/aegir
[22:18:29] <darthsteven> want to say anything?
[22:18:30] <anarcat> i think we're done!
[22:18:37] <anarcat> or late
[22:18:45] <anarcat> and since we're not allowed to be late, we're done :P
[22:18:48] <anarcat> ciao everyone
[22:18:52] <anarcat> thanks for attending :)
Weekly Scrum IRC Log: 2011-05-10
[12:14am] anarcat: darthsteven mig5 bgm ergonlogic mvc joestewart omega8cc sfyn shrop skwashd univate_ : scrum time!
[12:14am] anarcat: alright
[12:14am] anarcat: so for my part i haven't done much aegir work in the last week
[12:14am] anarcat: i was swamped with sysadmin duties and other koumbit stuff
[12:15am] anarcat: i am trying to fix that damn d7 segfault problem we're having and i'm stuck
[12:15am] anarcat: i will try to find sometime this week to finally setup the new jenkins server and start to work on provision unit tests
[12:15am] anarcat: sfyn something else to add to our scrum?
[12:15am] anarcat: omega8cc ?
[12:15am] omega8cc: I'm still busy writing some Nginx related updated docs for c.a.o when I have some rare free minutes, and in the meantime Barracuda and Octopus have been finally moved to d.o, yay!
[12:15am] omega8cc: See http://drupal.org/project/barracuda and http://drupal.org/project/octopus, so we can co-operate better now.
[12:15am] anarcat: yay!
[12:16am] anarcat: awesome
[12:16am] omega8cc: Next week I plan to publish the docs I collected so far. That is it for me.
[12:16am] sfyn: Since last I came I started rolling out releases for uc_hosting
[12:16am] sfyn: I am working on the next one this week and then I am porting the puppy to D7
[12:17am] sfyn: And implementing remote storefronts, which means I am going to start working on services
[12:17am] anarcat: awesome
[12:17am] Pathin left the chat room. (Ping timeout: 252 seconds)
[12:17am] sfyn: cmcintosh has convinced me, along with 8 of his friends, to go with servicesapi
[12:17am] sfyn: also
[12:17am] anarcat: it seems our gsoc student is missing, but he's going to work on the d7 port of hostmaster, so you may want to coordinate with him on that
[12:18am] sfyn: quota api, and particularly the sites implementation of it could use some love, and i've noticed that platforms being locked prevent you from changing anything on a site at all
[12:18am] sfyn: anarcat: what is his nick on irc?
[12:18am] anarcat: sfyn: he's sethvincent
[12:18am] Pathin joined the chat room.
[12:18am] sfyn: anarcat: and maybe you could fire over his email
[12:18am] anarcat: about platforms being locked - yeah, another user reported something like this recently
[12:18am] anarcat: sfyn: not sure i have his email, lemme check
[12:19am] sfyn: thanks
[12:19am] sfyn: one last thing
[12:19am] anarcat: done
[12:19am] sfyn: this discussion: http://community.aegirproject.org/discuss/aegir-and-phpmyadmin
[12:20am] sfyn: has a question about deploying phpmyadmin on slave servers, would be cool if someone could chip in with experience with those
[12:20am] anarcat: note that i haven't followed the issue queue or the community site in about a week, so i'm a bit lagging
[12:20am] Pasqualle left the chat room. (Ping timeout: 240 seconds)
[12:20am] sfyn: anarcat: received and thanks
[12:20am] anarcat: i see
[12:21am] Artusamak is now known as Artusamak_afk.
[12:21am] anarcat: not sure i have time for this right now, but i would just set it up as a separate vhost
[12:21am] anarcat: and no need to set it up on all servers...
[12:21am] anarcat: anyways, we need some integration there so that mysql servers get configured in PMA
[12:21am] anarcat: i think therE's an issue for that
[12:21am] anarcat: anyone else for the scrum?
[12:23am] • sfyn chirps like a cricket
[12:24am] mvc: did i mention my pet drush module here lately?
[12:24am] mvc: http://drupal.org/sandbox/mvc/1142496
[12:24am] anarcat: mvc: i don't think you did!
[12:24am] anarcat: sweet
[12:24am] mvc: it reports which projects (as in groups of modules) are in use by different sites in one or more platforms
[12:25am] anarcat: good work
[12:25am] mvc: as soon as i get around to testing it with d5 and d7 i'll move it out of the sandbox
[12:25am] anarcat: isn't status-report a bit confusing as a command name? i would expect that command to give me similar results than the status report page in the drupal site...
[12:26am] anarcat: otherwise, cool
[12:26am] omega8cc: there is some promising contrib to review, if you can help
[12:26am] Pathin left the chat room. (Quit: Lost terminal)
[12:27am] obrienmd: mvc: really intersted in testing that, will do so ASAP... agree w/ anarcat on the command name though
[12:27am] omega8cc: see http://drupal.org/node/1144968
[12:27am] FatGuyLaughing left the chat room. (Quit: FatGuyLaughing)
[12:29am] anarcat: alright
[12:29am] anarcat: oh
[12:29am] anarcat: another thing
[12:29am] anarcat: koumbit is going to make a presentation at drupalcon this year
[12:29am] anarcat: i would have loved to talk with darthsteven about this, i guess we can catch up later this week
[12:29am] anarcat: but basically, we're going to send the proposal soon, the deadline is the 15th, iirc
[12:30am] anarcat: so there we go
[12:30am] anarcat: i think that covers it for our scrum
[12:30am] anarcat: thanks everybody for attending
[12:30am] anarcat: and see you next week!
[12:30am] • anarcat scrum over!
[12:30am] omega8cc: ciao
[12:30am] sfyn: salut
[12:30am] szczym: see you
Weekly Scrum IRC Log: 2011-05-24
[22:00:11] * anarcat SCRUM TIME!
[22:00:27] <j0nathan> hi
[22:00:35] <anarcat> darthsteven / mig5 / bgm / grugnog / joestewart / mvc / omega8cc / sethvincent / / sfyn / shrop : scrum time!
[22:00:35] <hefring> anarcat: 2 days 8 hours ago <mig5> tell anarcat I don't know if one can export. I am happy to copy-and-paste, can you make me an admin of the system when you get a chance
[22:00:38] <anarcat> hallo :)
[22:00:50] <anarcat> mig5: sure will!
[22:00:54] <anarcat> so i'll start by saying
[22:00:54] <omega8cc> hello!
[22:00:54] <hefring> que tal
[22:00:55] <anarcat> hi!
[22:00:55] <hefring> hola
[22:00:56] <anarcat> :)
[22:01:09] <anarcat> i've got a few thinsg to touch base on
[22:01:18] <anarcat> darthsteven: we need to talk about drupalcon
[22:01:31] <anarcat> darthsteven: we'd like to see if you could participate in the training
[22:01:40] <anarcat> also
[22:01:53] <anarcat> i'll be going to the drush code sprint in boston 27-28 of june
[22:02:04] <anarcat> and i'll probably be around boston during the weekend too for the design camp
[22:02:12] <anarcat> so if anybody wants to hang out or say hi
[22:02:13] <anarcat> hi :)
[22:02:18] <anarcat> what else
[22:02:35] <anarcat> i'll look in the sites security issues this week - the stuff about sites being visible by anonymous users
[22:02:50] <anarcat> i know that sfyn is working on the uc_hosting module real hard, and got a 1.0 release out (or almost)
[22:03:11] <anarcat> we're looking at creating a d7 ubercart frontend that could talk to the aegir backend using cmcintosh's services_api stuff
[22:03:14] * henk_ has quit (Remote host closed the connection)
[22:03:22] <anarcat> so i think that's it for me!
[22:03:33] <anarcat> deadline for drupalcon votes is tomorrow, please vote for our sessions!
[22:03:50] <anarcat> http://london2011.drupal.org/conference/proposed-sessions?track=All&expe...
[22:03:54] <anarcat> anyone else?
[22:04:38] <sethvincent> i will officially start coding for gsoc today!
[22:04:50] <anarcat> yaaay! :)
[22:04:57] <anarcat> sethvincent: so what do you need help on now?
[22:05:07] <sethvincent> i'll be getting started by making a branch in my snadbox to test out the relevant patches in the issue queue
[22:05:20] * peterm95018 has quit (Quit: peterm95018)
[22:05:24] <anarcat> cool
[22:05:34] <anarcat> take a look at this one, i think that's a key issue: http://drupal.org/node/517620
[22:06:11] <sethvincent> right, that seems like the main one i need to work with
[22:06:20] <anarcat> okay
[22:06:27] <anarcat> if you need any help, please do let me know
[22:06:53] <anarcat> sethvincent: anything else?
[22:06:54] <sethvincent> i figure once i start testing out patches, doing a test run through coder upgrade and such things i'll have a list of questions i could use help with
[22:06:55] <anarcat> anyone else?
[22:07:02] <anarcat> awesome
[22:07:02] <sethvincent> that's about it for me.
[22:07:05] <anarcat> sethvincent: ping me any time
[22:07:07] <anarcat> as an aside, if anybody hangs around the channel wants some training for aegir, please look at http://london2011.drupal.org/page/aegir-hosting-system-deep-dive
[22:07:09] <sethvincent> ok. thanks
[22:07:28] <anarcat> darthsteven / mig5 / josh_k / omega8cc / shrop : anything to add to our weekly scrum?
[22:07:54] <anarcat> i can add for myself that i will be reviewing the patch queue tomorrow and try to get some of those commits in, and who knows - get 1.2 out the door? :)
[22:07:59] <joestewart> I posted a couple of simple patches that should be easy to review.
[22:08:06] <josh_k> I am currently working a small side project to try and get universities to use aegir
[22:08:15] <anarcat> awesome!
[22:08:21] <anarcat> double-awesome! ;)
[22:08:22] <josh_k> if it works, it should mean more users and hopefully more support (of the good kind)
[22:08:26] <josh_k> :)
[22:08:29] <anarcat> great :)
[22:08:35] <anarcat> joestewart: thanks for the patches
[22:09:06] <anarcat> alright, anyone else?
[22:10:15] <anarcat> omega8cc: ?
[22:10:18] <anarcat> maybe?
[22:10:48] <omega8cc> not much this week, still working on new docs for aegir/nginx
[22:10:59] <omega8cc> plus some video maybe
[22:11:04] <anarcat> good
[22:11:56] <anarcat> alright
[22:12:05] <anarcat> so i think that does it for our scrum, thanks everyone for attending!
[22:12:20] <anarcat> if anyone has anything to add, i'll stick around another 3 minutes, but in the meantime i'll upload the log to the site
[22:12:21] <omega8cc> ciao
Weekly Scrum IRC Log: 2011-05-31
[22:00:12] * anarcat SCRUM TIME!
[22:00:19] <anarcat> so welcome to our weekly scrum and blablablablabla :)
[22:00:26] <anarcat> I WANT TO RELEASE 1.2! :)
[22:00:42] <Aciid> more aegir harder faster stronger
[22:00:43] <anarcat> i've been itching to make a new release, and since we have a few good patches in, i thought it'd be a good idea
[22:00:54] <anarcat> i could do it on thursday
[22:00:56] <sfyn> WOW THAT SOUNDS GREAT ANARCAT! Is that the release we only talk about in all-caps?
[22:01:00] <anarcat> if nobody freaks out until then
[22:01:07] <anarcat> sfyn: no, it's not. :)
[22:01:22] * bwood has joined #aegir
[22:01:27] <anarcat> i've been mostly focused on working on the stable branch so far
[22:01:35] <anarcat> but i merged stuff around
[22:01:42] <anarcat> i wonder where our gsoc student is
[22:02:12] <anarcat> i'll be reviewing the patches sitting in the queue, so if you want to submit a patch for 1.x, now is a good time to push it up to RTBC
[22:02:19] <anarcat> and that's about it for me!
[22:02:31] <anarcat> anyone else got something to announce here?
[22:02:43] <sfyn> nope
[22:02:49] <anarcat> oh and i'll be working on the dupe client/site visible to anon stuff too this week
[22:02:57] * FransK has quit (Ping timeout: 258 seconds)
[22:03:23] <anarcat> since nobody is saying anything, i'll just bring something on a completely unrelated note: http://anarcat.koumbit.org/2011-05-23-construire-une-antenne-quad-en-ville
[22:03:33] <mig5> morning
[22:03:41] <anarcat> i'm building a huge ham radio antenna, 15'x15'x14', 11' in the air
[22:03:43] <anarcat> heey mig5 !
[22:03:46] <sfyn> anarcat: wow that really is very unrelated
[22:03:50] <mig5> omg i woke up in time for a scrum
[22:03:56] <jmcclelland> anarcat: i only now saw your response to http://drupal.org/node/1083366. I'd like to follow up and see if we can come up with a reasonable strategy before your next release.
[22:03:59] <anarcat> mig5: omgomgomg! :)
[22:04:24] <sfyn> Would this perhaps be a good time to mention I won't be going to London, and so Koumbit will need a third aegir dev interested in participating in our training?
[22:04:27] <anarcat> mig5: i changed the DNS records as you asked
[22:04:43] <anarcat> sfyn: i already mentionned this to darthsteven, we'll see where it goes
[22:04:46] <sfyn> sl;ash helping us to deliver our training
[22:04:49] <mig5> thanks anarcat. darthsteven has sudo on that box now so there's more hands on deck
[22:04:53] <sfyn> anarcat: ah, ok, thanks
[22:05:07] <anarcat> we're having trouble getting him to sync in during the scrum these days, maybe we can retune our schedule...
[22:05:29] <anarcat> mig5: great!
[22:05:46] <anarcat> jmcclelland: i'm not sure we can get that in 1.x, actually - it seems like this would change the API in a significant way...
[22:05:53] <anarcat> jmcclelland: but we can certainly talk about this :)
[22:05:56] <anarcat> and at least get a fix in 2.x
[22:06:04] <jmcclelland> ok - i understand.
[22:06:49] <anarcat> jmcclelland: but yeah, thanks a lot for getting involved, that issue is ... hard :)
[22:06:58] <anarcat> alrighty
[22:07:02] <anarcat> anyone else wants to jump in?
[22:07:08] <anarcat> doing cool aegir stuff?
[22:07:13] <anarcat> building crazy ham radio antennas? :)
[22:07:25] <mig5> hehe
[22:07:37] <sfyn> I am working on an extension to provision to provision ham radio antennaes
[22:07:38] <mig5> i've nothing else to add, i only worked on the jenkins machine since coming back from overseas
[22:07:58] <anarcat> sfyn: you're so lying :P
[22:08:11] <sfyn> anarcat: no it should be ready by quarter 5, 2019
[22:08:32] <anarcat> ok cool
[22:08:34] <anarcat> mig5: okay!
[22:08:46] <anarcat> mig5: i am not sure where to go with that other vserver now
[22:08:53] <anarcat> i've been busy with other stuff so i couldn't take care of that
[22:09:22] <anarcat> mig5: what do you think of a 1.2 thursday?
[22:09:31] <anarcat> or friday in your timezone :)
[22:09:39] <anarcat> (iirc)
[22:10:45] * bwood has quit (Quit: bwood)
[22:10:49] <mig5> should be ok, i'll be 'around'
[22:10:57] <mig5> what other vserver are you referring to
[22:11:24] <anarcat> mig5: the other jenkins server?
[22:11:27] <mig5> the original jenkins machine is my own jenkins machine that runs stuff on lots of my servers, so i've just dropped those old aegir jobs since migrating them
[22:11:47] <anarcat> didn't you setup a separate jenkins server for me to play with?
[22:11:49] <mig5> the reason we moved it was so i could give you sudo on a machine without exposing my evil little secrets :)
[22:11:56] <mig5> the new jenkins server is for you to play with
[22:12:06] <anarcat> oh, so that new ip is excatly that!
[22:12:07] <anarcat> i see
[22:12:11] <mig5> yep
[22:12:12] <anarcat> so you migrated all the jobs!
[22:12:13] <anarcat> awesome
[22:12:21] <mig5> yes so go crazy :)
[22:12:23] <anarcat> okay, then i guess that's one thing less on my list :)
[22:12:24] <anarcat> great!
[22:12:30] <mig5> i think we intended on not building 'cloud' machines but doing aegir installs directly on that box from git
[22:12:33] <anarcat> one thing i want to do is autobuild debian packges
[22:12:40] <mig5> oooh yes
[22:12:40] <anarcat> yeah that too
[22:12:44] <anarcat> do we still do the cloud thing?
[22:12:48] <mig5> yes for now
[22:12:55] <anarcat> okay so here's my idea
[22:13:00] <anarcat> we can chain jobs right?
[22:13:03] <mig5> yep
[22:13:05] <anarcat> so we have a job that builds the package
[22:13:11] <anarcat> another that installs it!
[22:13:15] <anarcat> dumb and simple :)
[22:13:17] <mig5> perfect
[22:13:32] <anarcat> oh wait, there's another one (duh) - another one removes the package! :)
[22:13:34] <mig5> and we can chain an install + upgrade too
[22:13:40] <anarcat> totally
[22:13:45] <anarcat> so we can have a builder for the last release
[22:13:56] <anarcat> then use that package to install then chain with the upgrade
[22:14:13] <anarcat> the intricacies of this are just scary for a jenkins newbie like me, but it looks promising
[22:14:19] <anarcat> i tried the openid plugin again
[22:14:25] <anarcat> and it failed on me again
[22:14:37] <mig5> chaining seems pretty straightforward in jenkins: and it's smart enough to know whether to not build if the previous job failed, etc
[22:14:37] <anarcat> problem is - i became the openid_provider maintainer now, so i have no excuse left :P
[22:14:40] <mig5> hrm
[22:14:40] <mig5> haha
[22:15:15] <anarcat> kinda sucks :P
[22:15:43] <mig5> fuckity fuck :)
[22:15:48] <mig5> too funny
[22:15:50] <anarcat> yeah
[22:15:55] <anarcat> so anyways
[22:16:00] <anarcat> i think that's it for our scrum folks!
[22:16:03] <anarcat> thanks for attending!
[22:16:11] <mig5> cheers
Weekly Scrum IRC Log: 2011-06-07
[15:00] * anarcat SCRUM TIME!
[15:00] <@anarcat> welcome everybody
[15:00] <+sfyn> hello
[15:00] <@anarcat> so this is our weekly scrum
[15:00] <j0nathan> halo
[15:00] <@anarcat> and i'll be your host, jolly-o-anarcat
[15:00] <@anarcat> this week
[15:00] <@anarcat> i want to release 1.2
[15:00] <@anarcat> i know i said that last week
[15:00] <@anarcat> but i failed
[15:00] <@anarcat> so i'll try again :)
[15:00] <@anarcat> if you guys have patches to review or commit, now is a good time
[15:00] <@anarcat> i'll do a sprint tomorrow and try to release thursday
[15:01] <@anarcat> also
[15:01] <@anarcat> koumbit wants to blog about "cool things done with aegir" - we have a few on our end
[15:01] <@anarcat> but if you guys have good examples, bring it!
[15:01] <@anarcat> you can send sample sites or blurbs or text or whatever to me on irc here or by email at anarcat@koumbit.org
[15:01] <@anarcat> what else what else
[15:02] <@anarcat> i think that's it for me, other than the fact that i'm probably supposed to harrass our gsoc student here
[15:02] <@anarcat> sethvincent: prod!
[15:02] <+sethvincent> hello!
[15:02] <@anarcat> prod prod :)
[15:02] == dunkoh [~dunkoh@MW-ESR1-72-49-37-45.fuse.net] has joined #aegir
[15:02] <@anarcat> hey seth, how's it going
[15:02] <@anarcat> can you give us a little progress update?
[15:02] <+sethvincent> sorry. classes here have just finished, so i'll be more active starting this week.
[15:03] <+sfyn> yay
[15:03] <@anarcat> yay! :)
[15:03] <+sfyn> !
[15:03] <+sethvincent> i've been working on porting eldir first.
[15:03] <+sethvincent> made a few initial commits of that work
[15:03] <+sethvincent> applied the 'turn features into features' patch and have been looking through the changes that introduces
[15:04] <+sethvincent> and that's about it
[15:04] <@anarcat> okay, those commits don't seem to be pushed on drupal.org, please do so when you can!
[15:04] <@univate> I am willing to help out with some of that "features" stuff if you need it
[15:04] <@anarcat> good work
[15:04] <@anarcat> awesome!
[15:04] == dunkoh [~dunkoh@MW-ESR1-72-49-37-45.fuse.net] has quit [Client Quit]
[15:04] <@anarcat> univate: please do! i understand you had good contributions there
[15:04] * sfyn raises his hand
[15:04] <@anarcat> in fact, if you want to help sethvincent by reviewing and merging his patches in, that's great
[15:04] <@anarcat> sfyn: go right ahead
[15:04] <@anarcat> i think seth was done
[15:04] == jerryitt [~jerry@ip-78-137-175-99.dsl.digiweb.ie] has joined #aegir
[15:05] <+sethvincent> cool. so far all the stuff i'm working on is here: http://drupal.org/sandbox/seth.vincent/1140244
[15:05] <+sfyn> not much done, but I have started working on the servicesAPI imp from the hostmaster_plus module
[15:05] <+sfyn> Bit of a mess, but I am hosting that work on github for now
[15:05] <+sfyn> https://github.com/sfyn/hosting_services
[15:05] == AquaticDisorder [~AquaticDi@cpc2-chwo3-0-0-cust50.perr.cable.virginmedia.com] has joined #aegir
[15:05] <+sfyn> The big news - the module can now be installed
[15:05] <@univate> I have a couple a major projects that are planning to use aegir, so hoping to have a a bit more time focused on this
[15:05] <@anarcat> sethvincent: i see... the url linked from the gsoc page is wrong: http://www.google-melange.com/gsoc/proposal/review/google/gsoc2011/sdv_3333/1#
[15:05] <+sfyn> I am waiting to hear back from cmcintosh - the original author - before publishing to d.org
[15:06] <+sethvincent> anarcat: oh! i'll fix that
[15:06] <@anarcat> sfyn: why isn't that on drupal.org?
[15:06] <@anarcat> oh nevermind :)
[15:06] <@anarcat> sfyn: i think you should go ahead and publish
[15:06] <+sfyn> I agree
[15:06] <+sfyn> if mr. mac doesn't get back to me by friday I will
[15:06] <@anarcat> ok
[15:07] <@anarcat> great
[15:07] <+sfyn> YOU HEAR THAT CMCINTOSH?
[15:07] <@anarcat> SFYN: I DON'T THINK HE'S HERE!
[15:07] <@anarcat> ;)
[15:07] <@anarcat> alright, enough yelling :)
[15:07] <+sfyn> maybe he will read the scrum log later
[15:07] == FatGuyLaughing [~FatGuyLau@65.64.83.127] has quit [Quit: FatGuyLaughing]
[15:07] <@darthsteven> my turn
[15:07] <@anarcat> sfyn, if you're done, i think we can move on to mig5 or darthsteven
[15:07] <@anarcat> alright :)
[15:07] * sfyn clams up
[15:08] <@mig5> nothing from me this week i'm afraid
[15:08] <@darthsteven> so, this week I've been reviewing a few issues and committing some of my patches to the 7.x branch and 6.x too
[15:08] <@mig5> i hear the distribution relesae node issue was fixed
[15:08] <@darthsteven> thinking about things to do with cron
[15:08] <@anarcat> darthsteven: cool
[15:08] <@anarcat> mig5: yes, at last!!!
[15:08] <@darthsteven> I'm going to hit documentation hard over the next few weeks
[15:09] <@darthsteven> to live up to my 'documentation maintainer' title :)
[15:09] <@anarcat> cool :)
[15:09] <@anarcat> that's great!
[15:09] <@darthsteven> I'd also like to get api.aegirproject.org sorted out
[15:09] <@darthsteven> cause it's a bit of a mess at the moment
[15:09] <@anarcat> darthsteven: we should really move the api site to our aegir server, that would fix a lot of shit
[15:09] <@anarcat> yeah
[15:09] <@anarcat> ergonlogic was proposing we move to d7 too
[15:10] <@darthsteven> we can discuss in the issue...
[15:10] <@anarcat> yeah
[15:10] <@darthsteven> that's it from me
[15:10] == ipwa [~ipwa@190.222.35.202] has joined #aegir
[15:10] <@anarcat> alright
[15:10] <@darthsteven> anyone else?
[15:10] <@anarcat> mig5: anything to add?
[15:10] <@mig5> nope
[15:10] <@anarcat> otherwise i think we're kind of done here
[15:10] <@anarcat> thanks everyone for attending, and see you on the other side of 1.2 :)
[15:10] <@anarcat> oh
[15:10] <+sfyn> cya all next week
[15:10] <@anarcat> another thing
[15:11] <@anarcat> i am trying to remove aegir from ubuntu
[15:11] <@anarcat> because it currently ships with 0.3
[15:11] <+sfyn> oops
[15:11] <@anarcat> which is confusing some people
[15:11] <@anarcat> and is blocking the drush backport
[15:11] <@anarcat> but that seems to be hard
[15:12] <@anarcat> if anyone wants to help in the ubuntu effort, that would be apprecitated: https://bugs.launchpad.net/ubuntu/+source/aegir-provision/+bug/793567
[15:12] <@anarcat> and https://bugs.launchpad.net/ubuntu/+source/drush/+bug/755169
[15:12] <@anarcat> so that's about it
[15:12] <@anarcat> thanks again everyone
Weekly Scrum IRC Log: 2011-06-21
21:59 <@anarcat> hey 21:59 <@anarcat> gooood morning vietnam! 22:00 <@mig5> HELO 22:00 <@anarcat> alright 22:00 * anarcat SCRUM TIME! 22:00 <@anarcat> gosh this is early 22:00 <@anarcat> so welcome everyone 22:00 <@darthsteven> heh 22:01 <@anarcat> i'm your sleepy host, unless another timezone feels less jetlagged :) 22:01 <@mig5> getting sleepy :) 22:01 <@darthsteven> I'm your less sleepy host then 22:01 <@anarcat> yay! :) 22:01 <@darthsteven> Welcome to the scrum for 21 Jun 2011 22:01 <@darthsteven> Who wants to speak first? 22:01 <@darthsteven> anarcat? 22:01 <@anarcat> i can try! :) 22:02 <@anarcat> so i haven't published 1.2, but this time i have a good excuse 22:02 <@anarcat> we're really almost there - i want to test the debian packages first 22:02 <@anarcat> i was able to make an autobuilder in jenkins 22:02 <@anarcat> so jenkins now has a pgp key and can build packages. both from the debian branch and from the 1.x branch 22:02 <@anarcat> https://drupal.org/node/1183926 22:02 <@darthsteven> cool 22:03 <@anarcat> there are a good few questions in there that i'd like feedback about 22:03 <@anarcat> esp. from mig5 22:03 <@mig5> awesome, you are now our CI ninja :) 22:03 <@anarcat> crap :) 22:03 <@darthsteven> anything else anarcat? 22:03 <@anarcat> well, there's this daemon thing :) 22:04 <@anarcat> i played with darthsteven's queue runner, and it's awesom 22:04 <@anarcat> we're running it in production, working very well, got people impressed 22:04 <@anarcat> i want to merge it in 2.x 22:04 <@anarcat> and port our dispatcher to the d7 queue system 22:04 <@anarcat> i think we should make queue-runner 1.1 22:04 <@anarcat> i think that's about it for me! 22:05 <@darthsteven> mig5? 22:05 <@darthsteven> (thanks anarcat) 22:05 <@anarcat> welcome! 22:05 <@mig5> nothing from me this week, i need to catch up on the jenkins changes 22:05 <@darthsteven> okay, cool 22:05 <@darthsteven> I'll go next... 22:06 <@darthsteven> I looked at the daemon for running tasks with anarcat 22:06 <@darthsteven> resulting in a 1.1 release, which anarcat hasn't noticed :) 22:06 <@anarcat> haha cool 22:06 <@darthsteven> We also got the api site sorted last week, though mostly not down to me 22:06 <@darthsteven> which is ace! 22:07 <@mig5> nice 22:07 <@darthsteven> I added a new hook to enable friendly names for servers in lists of servers 22:07 <@darthsteven> and released a module that uses the hook 22:07 <@darthsteven> http://drupal.org/project/hosting_server_titles 22:08 <@darthsteven> I think I added a ton of documentation the week before too 22:08 <@darthsteven> which was nice 22:08 < chouchou> Hello, 22:08 <@anarcat> cool 22:08 <@darthsteven> but that's it from me 22:08 < chouchou> pls i have an issue, 22:08 <@anarcat> note that i documented the 2.x api changes here http://community.aegirproject.org/upgrading/path 22:08 < chouchou> first, i wouuld like to know how do I change the drush gid download source ? 22:08 <@darthsteven> chouchou: We're in the middle of a scrum, please hold for the end, then you can ask 22:08 <@anarcat> i think we should keep that page in mind when we change the API 22:08 <@darthsteven> anyone else have anything for the scrum? 22:09 <+omega8cc> I finally started publishing some nginx related docs on www.omega8.cc and will extract from there to re-post on c.a.o handbook, also some new nginx patches are in testing and should be sumitted for review shortly, we are sponsoring also Aegir session at http://drupaldesigncamp.net next week! 22:09 <@darthsteven> awesome 22:09 <@mig5> props to shrop who has done great work on the community site sorting some stuff out 22:09 <@darthsteven> omega8cc: anything else? 22:09 <@anarcat> all good 22:09 <+omega8cc> that is it from me, i think 22:10 <@anarcat> ah, note that i'll be at the drush test sprint next week, so i will probably miss the scrum 22:10 <@anarcat> if you guys have specifics of things i should work on with drush, let me know! 22:11 <@darthsteven> anyone else have anything to add? 22:11 <@darthsteven> I'll give it a minute... 22:11 <@anarcat> well, i'd like to chat quickly with mig5 about the jenkins stuff 22:11 <@anarcat> mig5: did you read up on the issue? 22:11 <@mig5> this one? https://drupal.org/node/1183926 22:12 <@anarcat> mig5: upy 22:12 <@anarcat> yup* 22:12 <@anarcat> basically, we need to decide this: 22:12 <@anarcat> 1. do we install/remove/update/whatnot on the jenkins server? (be scared) 22:12 <@anarcat> 2. do we install using dpkg -i or do we upload to debian.koumbit.net's unstable repository and use apt-get? (the latter has the disadvantage of having to wait a undetermined delay for repo to catchup, but has the upside of broadening the test capabilities) 22:12 <@anarcat> i am thinking more and more of this as a build bot, with packges uploaded to debian.koumbit.net/unstable 22:13 <@anarcat> and then everyone can test them, including jenkins or "the cloud", from jenkins 22:13 <@mig5> 1. is fine with me but i was already scared :) but as far as I can tell, it's the only way to sanely chain tasks.. (as opposed to an isolated cloud-generated VPS) 22:14 <@mig5> the fundamental thing i want to do with theses tests, is pretend like we are a user 22:14 <@anarcat> well that's the thing, if we upload to debian.k.n, then we can run install/upgrade/whatever independently... 22:14 <@mig5> except with preseeding :) 22:14 <@mig5> so i think the apt-get is good 22:14 -!- bgm_ is now known as bgm 22:14 <@anarcat> ok 22:14 <@anarcat> mig5: do you know about jenkins's "tasks"? 22:14 -!- bgm [~bgm@2607:f748:1200:105::5] has quit [Changing host] 22:14 -!- bgm [~bgm@pdpc/supporter/active/bgm] has joined #aegir 22:14 <@mig5> no 22:14 <@anarcat> ok 22:14 <@mig5> i've only done build steps 22:15 <@anarcat> i tried to play with that, but i don't feel it's what we need, basically... 22:15 <@anarcat> it's a weird thing 22:15 <@anarcat> so okay, that's cool 22:15 -!- mode/#aegir [+v shrop] by ChanServ 22:15 <@anarcat> i think i'll try to setup fabric stuff to deploy debian packages remotely 22:15 <@anarcat> and of course upload to debian.k.n 22:15 <@mig5> deploy as in send to koumbit.d.n ? 22:15 <@mig5> er 22:15 <@mig5> ah 22:15 <@anarcat> and fix the preseed 22:16 <@mig5> so by deploy debian packages remotely you mean, build on fresh VPS? 22:16 <@mig5> as i am not against the former 22:16 <@anarcat> deploy as in ssh cloud.vps.foo 'apt-get install aegir; drush provision-tests ; apt-get remove aegir' 22:16 <@mig5> so long as we can clean up our mess on the jenkins server 22:17 <@anarcat> well, i'm worried about not being able to do that 22:17 < chouchou> are you free now? 22:17 <@mig5> my main worry about cloud.vps.foo is we may lose the ability to chain builds 22:17 <@anarcat> *and* about interop problems with jenkins itself 22:17 <@darthsteven> Anyone else have anything for the scrum? Otherwise we'll wrap it up there, and leave mig5 and anarcat to it... 22:17 <@anarcat> darthsteven: yup, ok 22:17 <@darthsteven>
Weekly Scrum IRC Log: 2011-06-28
[22:01:28] <darthsteven> scrum time!
[22:01:35] <darthsteven> who wants to go first?
[22:01:51] <darthsteven> me?
[22:01:55] <darthsteven> okay, if you insist
[22:02:09] <darthsteven> I've not done much since last week
[22:02:27] <darthsteven> Don't really have anything to report
[22:02:36] <darthsteven> that's it from me!
[22:02:43] <darthsteven> anyone else have anything to add?
[22:03:08] <mig5> i've done nothing this week either.. sadly I don't see things changing from that, either..
[22:03:16] <shrop> darthsteven: I hope to add more docs to community.aegirproject.org soon. Some ldap config info for Active Driectory
[22:03:29] <mig5> nice
[22:03:34] <darthsteven> awesome
[22:03:42] <darthsteven> anyone else?
[22:04:15] <omega8cc> I fixed a few bugs in the Nginx config, submit even more patches for review and also discovered we don't support D7 properly yet: http://drupal.org/node/1196034
[22:04:15] <hefring> http://drupal.org/node/1196034 => Paths to images are broken in D7 after site clone/rename => Provision, Code, critical, needs work, 2 comments, 1 IRC mention
[22:05:08] <darthsteven> oh hello, that's a bit of a showstopper
[22:05:31] <darthsteven> anyone else?
[22:05:45] <omega8cc> and that is it from me, I think, I plan to submit even more patches next/this week
[22:06:04] <darthsteven> Thank you omega8cc
[22:06:13] <darthsteven> anyone else have anything to add/discuss?
[22:06:36] * mrconnerton has joined #aegir
[22:06:41] <omega8cc> so, what about 1.2?
[22:07:05] <omega8cc> are we waiting for anarcat probably?
[22:07:27] <darthsteven> I think that anarcat was working to get the debian packages built and tested automatically before releasing 1.2
[22:07:36] <darthsteven> and we need to fix the issue you found
[22:07:43] <omega8cc> yeah
[22:08:20] <darthsteven> Well, we'll end the scrum there I think
[22:08:27] <darthsteven> thanks for coming
[22:08:33] <darthsteven> logs?
[22:08:33] <hefring> yeah, ok, i admit, i'm logging this channel in http://hefring.mig5.net/bot/log/aegir - ask me about log bookmark too ;)
[22:08:45] <darthsteven> hefring: log bookmark?
[22:08:45] <hefring> http://hefring.mig5.net/bot/log/aegir/2011-06-28#T88506
Weekly Scrum IRC Log: 2011-07-05
[22:01:04] <mig5> scrum time
[22:01:12] <omega8cc> hello!
[22:01:12] <mig5> anarcat sends his apologies for this week
[22:01:25] <mig5> but I hear he had a good time at the drush sprint and got some good work done, so hopefully we'll hear more about that soon
[22:02:03] <mig5> me, i once again didn't do anything.. oh, I did add to the jenkins deb install of aegir to uninstall and purge the packages before destroying
[22:02:13] <mig5> which is a final good test of those packages
[22:02:18] <halcyonCorsair> omega8cc: i'm trying a barracuda/octopus upgrade before i try again
[22:02:40] <mig5> i think we are well overdue for a 1.2 release, I am conscious there are a lot of patches from omega8cc for review (I think?) so we should get those in
[22:02:53] <halcyonCorsair> omega8cc: but i was having trouble replicating the site installation commands, i think there might have been some arguments i was missing?
[22:02:55] <omega8cc> halcyonCorsair: please wait 15 minutes, we have scrum time
[22:02:56] <mig5> that's it from me, over to darthsteven and omega8cc
[22:03:27] <darthsteven> sure
[22:03:52] * Harley has joined #aegir
[22:03:56] <darthsteven> I got the Jenkins deb install of aegir working
[22:04:07] <mig5> epic
[22:04:13] <darthsteven> and worked on getting notification integration with Aegir
[22:04:26] <darthsteven> http://drupal.org/project/hosting_notifications
[22:04:42] <mig5> oh nice. i saw somewhere a sandbox link but it had disappeared
[22:04:47] <darthsteven> So I now get push notifications on my iPhone when Aegir tasks fail
[22:04:51] <darthsteven> which is nice
[22:04:58] <darthsteven> write-up coming soon
[22:05:02] <mig5> nice. Notifo?
[22:05:11] <darthsteven> that's it from me
[22:05:25] <darthsteven> (using prowl, but whatever messaging module supports is possible)
[22:05:36] <mig5> gotcha
[22:06:23] <mig5> omega8cc, anything to add? other than 'commit my patches you bastards' :)
[22:06:42] <omega8cc> ok, so I want to attack that weird critical today: http://drupal.org/node/1196034 and it lock all our upgrades schedule currently, and then will submit next round of improvements to the Nginx config plus - yay! Aegir UI patches!
[22:06:42] <hefring> http://drupal.org/node/1196034 => Paths to images are broken in D7 after site clone/rename => Provision, Code, critical, needs work, 3 comments, 3 IRC mentions
[22:07:52] <mig5> cool!
[22:07:55] <omega8cc> and that is it from me, but the UI changes are long overdue and I want to share them finally
[22:09:09] <mig5> omega8cc: i wonder if that patch of yours needs to be applied to the deploy_7.inc file?
[22:09:13] <mig5> as opposed to deploy.inc
[22:09:18] <mig5> but i've forgotten how it all works
[22:09:54] <mig5> don't think it would change that result anyway
[22:09:57] <omega8cc> mig5: that is exactly what I think, it misses d7 specific stuff there completely
[22:10:04] <mig5> strange
[22:10:08] <omega8cc> yeah
[22:10:15] <mig5> anyway - anyone else got something to add to the scrum? ergonlogic?
[22:10:25] <joestewart> not directly on aegir but I posted two patches to drush_make that are useful for aegir. one that adds ability to set options in the makefile - http://drupal.org/node/1206340 the other updated a patch that allows setting working-copy per project - http://drupal.org/node/958844
[22:10:25] <hefring> http://drupal.org/node/1206340 => introduce an options array in the root level of the makefile => Drush Make, Code, normal, needs review, 0 comments, 1 IRC mention
[22:10:26] <hefring> http://drupal.org/node/958844 => PATCH: Allow [working-copy] in makefile as well as on commandline => Drush Make, Code, normal, needs review, 18 comments, 1 IRC mention
[22:10:42] <mig5> nice, I saw the working-copy patch update
[22:11:08] <mig5> also see Owen's Drush sprint overview here for those interested: http://civicactions.com/blog/2011/jun/30/drush_code_sprint_report
[22:11:29] * CraHan is now known as CraHan_out
[22:11:52] <mig5> ok, scrum done I think, and i'll do the log
[22:11:56] <mig5> thanks all
[22:12:04] <omega8cc> thanks!
Weekly Scrum IRC Log: 2011-07-12
(08:01:34 AM) anarcat: heey scrum time (08:01:39 AM) anarcat: mig5 / darthsteven : scrum! (08:01:45 AM) anarcat: omega8cc / sfyn / shrop : scrum time (08:01:52 AM) anarcat: am i the only one awake? :) (08:01:59 AM) omega8cc: hello! (08:02:01 AM) ergonlogic: anarcat: not the *only* one (08:02:14 AM) ergonlogic: hi all (08:02:35 AM) anarcat: hello ergonlogic ! (08:02:50 AM) anarcat: so welcome to our weekly scrum (08:02:52 AM) anarcat: i don't have much to say (08:02:57 AM) anarcat: i've been away for a while (08:03:03 AM) anarcat: vacations, drush sprint, etc (08:03:12 AM) anarcat: i'll try to do a reportback of that great drush sprint soon (08:03:36 AM) anarcat: but basically, i worked on jenkins, debian packaging and cleaning up the drush bootstrap process to make real unit tests possible (08:03:49 AM) anarcat: so we can now test *functions* (instead of just commands) in the drush unit test framework (08:03:53 AM) anarcat: which will be useful for provision (08:03:54 AM) anarcat: now (08:04:01 AM) anarcat: how's 1.x doing? (08:04:14 AM) anarcat: i have on my tuesday todo (and had for a while) to release 1.2 (08:04:27 AM) anarcat: so i'm thinking to test the 1.x debian packages all over the place then just release the damn thing (08:04:53 AM) anarcat: how does that sound? (08:04:55 AM) omega8cc: anarcat: there are patches waiting in the queue (08:05:02 AM) anarcat: ah darn :) (08:05:05 AM) anarcat: omega8cc: your patches? :) (08:05:07 AM) omega8cc: ;) (08:05:10 AM) omega8cc: yep (08:05:13 AM) anarcat: hehe ok (08:05:17 AM) anarcat: that shouldn't be a big issue (08:05:26 AM) omega8cc: simple stuff (08:05:59 AM) anarcat: ok, so i'll merge that in before release (08:06:09 AM) omega8cc: thanks (08:06:09 AM) anarcat: so one last thing for me (08:06:17 AM) anarcat: i'll be going on a looong holiday around drupalcon (08:06:33 AM) anarcat: i'm leaving on august 3rd for europe, then going to drupalcon, then i'm gone for september (08:06:54 AM) anarcat: so that could stall dev of 2.x for a little longer, but i'll be back in full force after that (08:07:22 AM) anarcat: ah and before i forget: darthsteven: our hosting-queue-runner daemon disappeared here for the first time ever, not sure why, i'll try to track it down today (08:07:29 AM) anarcat: okay, anyone else has something to add here? (08:07:45 AM) ergonlogic: For my part, I posted an Aegir extension module on d.o: Hosting Profile Roles (08:07:50 AM) ergonlogic: ... and I've begun hacking on a couple more (08:07:55 AM) ergonlogic: And worked on a couple small patches to Drush (08:08:09 AM) ergonlogic: I've been testing .deb installs on some client VPSs, which has been going smoothly (08:08:12 AM) omega8cc: not much from me this week, besides fixed support for open public and vudeola (08:08:33 AM) omega8cc: err, videola (08:08:52 AM) ergonlogic: ooh, lullabot's published it now? (08:08:56 AM) omega8cc: yep (08:09:04 AM) ergonlogic: I'll have to check it out (08:09:14 AM) omega8cc: and merged my fix already (08:09:44 AM) Aciid: oh no, videola. I hope it doesn't catch on at the company that we should start offerin sites with it.. (08:09:56 AM) Aciid: videola's performance and seek capabilities are horrible (08:10:08 AM) Aciid: or maybe it's lullabot's CDN (08:11:28 AM) anarcat: alright, anyone else? (08:11:43 AM) ***anarcat notices our gsoc student missing (08:11:52 AM) anarcat: has he checked in at all while i was gone? (08:12:13 AM) omega8cc: I didn't notice (08:12:26 AM) anarcat: that's reallly too bad (08:12:32 AM) anarcat: i'll have to do an evaluation soon (08:12:57 AM) ergonlogic: oh, one other things (not to do with our GSoC student), Koumbit is now accepting donations in support of the Aegir Project (08:13:15 AM) ergonlogic: there's a link on the front page of community.aegirproject.org (08:14:29 AM) ergonlogic: if/when we receive any donations, we'll have to figure out how to disburse them (08:14:33 AM) anarcat: ergonlogic: yeah, mig5 wanted to talk about this, actually (08:14:55 AM) anarcat: ergonlogic: he was suggesting we (koumbit) get a rackspace cloud account and deal with the donations (08:14:59 AM) anarcat: where are the donations going now? (08:15:16 AM) ergonlogic: anarcat: Koumbit's PayPal acct (08:15:23 AM) anarcat: ok, good (08:15:46 AM) anarcat: alright (08:15:57 AM) anarcat: so if there's nobody else, that will be it for our weekly scrum! (08:16:13 AM) ergonlogic: I'll post to the community site (08:16:28 AM) omega8cc: thanks
Weekly Scrum IRC Log: 2011-07-19
22:00 <@anarcat> alright 22:00 <@anarcat> scrum time! 22:00 <+omega8cc> oh 22:01 <@mig5> I didn't know if it had moved time yet or what :) 22:01 <@anarcat> darthsteven: scrum! 22:01 -!- Irssi: #aegir: Total of 75 nicks [4 ops, 0 halfops, 4 voices, 67 normal] 22:01 <@anarcat> so hi 22:01 <@anarcat> i am not sure what to do about the time 22:01 <@anarcat> steven says he can't attend every week because it's too late 22:02 <@anarcat> mig5: and you seem also to have issue about i 22:02 <@mig5> not ideal for everyone, ever.. maybe we just tolerate a 'rotation' of devs who might not always be here 22:02 <@anarcat> t 22:02 <@mig5> maybe it helps that I never have anything to add... :/ 22:02 <@anarcat> well 22:02 <@anarcat> i will be on a holiday for two months 22:02 <@anarcat> so maybe we can flip that around somehow 22:03 <@anarcat> http://www.timeanddate.com/worldclock/meetingdetails.html?year=2011&mont... É 22:03 <@anarcat> ? 22:03 <@anarcat> 8hAM london, 5PM melbourne, midnight LA 22:04 <@mig5> we'll see if darthsteven complains about the early start :) 22:04 <@mig5> fine with me 22:04 <@darthsteven> Fine with me 22:04 <@anarcat> 2 am EDT 22:04 <@anarcat> anyways 22:04 <@anarcat> i don't know what to do 22:04 <@anarcat> and i wish somebody else would figure it out basically :P 22:05 <+omega8cc> fine with me too ^^ 22:05 <@mig5> it won't ever work until I quit :) 22:05 <@darthsteven> Could we rotate early and late on alternate weeks? 22:05 <@anarcat> we could! 22:05 <@anarcat> and really, we need to keep in mind we don't absolutely need to readjust for seth 22:05 <@anarcat> he was pretty clear about that 22:06 <@darthsteven> So long as it's not too confusing 22:06 <@anarcat> he can (and should) provide feedback in other ways 22:06 <@anarcat> the 8AM is kinda hard on some people here though 22:06 <@anarcat> so anyways 22:07 -!- Chipie [~Chipie@mue-88-130-106-140.dsl.tropolys.de] has quit [Quit: Leaving.] 22:07 <@anarcat> to be clear - i would rather see both of you guys (mig5 and darthsteven) than bring seth back in the loop and loose you :P 22:07 <@anarcat> so i suggest we stick to the schedule at least until i leave for europe 22:07 <@mig5> don't worry too much 22:07 <@anarcat> then you guys can organize as you want 22:07 <@mig5> sure 22:07 <@anarcat> which brings me to 22:08 <@anarcat> somebody should probably take over gsoc mentorship while i'm gone 22:08 <@anarcat> i may have *some* time to followup, but really not a lot 22:08 <@anarcat> i'll be traveling all over the place and I won't be able to prod seth 22:08 <@anarcat> and he needs prodding :P 22:08 <@anarcat> so if somebody could take that over, that would be cool 22:09 -!- redShadow[2V] [~redShadow@93-62-151-43.ip23.fastwebnet.it] has joined #aegir 22:09 <@anarcat> it's just sending an email from time to time to say "hey what's up" - he's usually pretty responsive 22:09 <@anarcat> i tried to encourage him to document hist stuff on the commmunity site 22:09 <@mig5> ok. I am probably doing the least all round so I will step up 22:09 <@anarcat> and we reoriented his roadmap 22:09 <@anarcat> which he needs to document 22:09 <@anarcat> mig5: awesome! 22:09 <@mig5> if you could brief me in an email on the current status, maybe cc him 22:09 <@anarcat> mig5: i can forward you the relevant communications 22:09 <@mig5> so i'm up to date 22:09 <@mig5> snap 22:09 <@anarcat> hehe 22:09 <@anarcat> cool 22:09 <@anarcat> so that will be it 22:10 <@anarcat> i will probably receive the "hey you need to evaluate this guy" at the end of the program, but we'll cross that bridge when we get there 22:10 <@anarcat> alright 22:10 <@anarcat> now 22:10 <@anarcat> i released 1.2 22:10 <@anarcat> whooohooo! 22:10 <@anarcat> http://community.aegirproject.org/1.2 22:10 -!- romaingar [5d197493@gateway/web/freenode/ip.93.25.116.147] has joined #aegir 22:10 <@mig5> thanks for that anarcat, it got a good response 22:10 <@anarcat> omega8cc found what seems to be a critical with the new cron code: https://drupal.org/node/1222208 22:11 <@anarcat> i tried to roll out a patch for it and it failed to work, and i'm totally puzzled as to why 22:11 <@darthsteven> Cool 22:11 <@anarcat> i don't see the same issue at all here 22:11 <@mig5> i wondered if it was nginx specific, caching HTTP results 22:11 <@anarcat> and i'm about to give up on it for now, as we don't see the bug at koumbit 22:11 <@anarcat> mig5: s/was/is/ 22:11 <@anarcat> the issue is still there 22:11 <@mig5> yeah. 22:11 <@anarcat> so i don't get it 22:11 <@anarcat> the original code *was* broken, but with the patch it should be fine 22:12 <@anarcat> http://drupalcode.org/project/hostmaster.git/commitdiff/f46c7ce 22:12 <@mig5> it would not be the first time i've seen nginx cache a 404 22:12 <+omega8cc> there is no caching there 22:12 <+omega8cc> none 22:12 <@mig5> 2 days is too much though 22:12 <@mig5> ok 22:12 <@darthsteven> Not sure I have time, but will try to look if I can 22:12 <+omega8cc> just one 403 error locks the queue 22:13 <@anarcat> omega8cc: and you did try the patch? 22:13 <@anarcat> omega8cc: do you see this watchdog entry? watchdog('hosting_cron', 'cron failed on site %site with error %error', array('%site' => $site->title, '%error' => $response->error), WATCHDOG_NOTICE); 22:13 <+omega8cc> yes, and reported failure there 22:13 <@anarcat> we'll need more juice to figure that one out 22:13 <+omega8cc> yes, it says 403, as before, for site taken offline for maintenance 22:14 <+omega8cc> this should be easy to reproduce 22:14 <@anarcat> ok 22:14 <@anarcat> so anyways, maybe darthsteven will look into it 22:14 <@anarcat> we can talk more about it after the scrum 22:14 <@anarcat> i monopolized the space too long :) 22:14 <@anarcat> otherwise yeah, release++ 22:14 <@anarcat> the debian packages kick major asses 22:15 <@anarcat> and we're deploying a bunch of vservers with those 22:15 <@anarcat> so it's fun 22:15 <@anarcat> that's about it for me, anyone else? 22:15 <@mig5> nothing here 22:16 <@anarcat> darthsteven? 22:16 <@anarcat> oh crap, that should be a continue, not a break 22:16 <@anarcat> i'm on acid 22:16 <@anarcat> ffs 22:17 <@mig5> ?? 22:17 <@mig5> ah 22:17 <@anarcat> anyways, i have a fix for the cron thing 22:17 <@mig5> haha 22:17 <@darthsteven> I've been on holiday, and not had time to do anything Aegir related 22:17 <@darthsteven> Will do something Aegir related in the next week for sure 22:17 <@darthsteven> Need to check up on our Drupalcon presentation with anarcat 22:18 <@anarcat> darthsteven: any specific time you want to do this? 22:18 <+omega8cc> hah, break does just that - breaks things! lol 22:18 <@darthsteven> anarcat: not fussy about a time 22:18 <@mig5> failed to fail failed :) 22:19 <@darthsteven> anarcat: soon though, as I think they need things from us? 22:20 <@anarcat> yeah, i've been kind of sloppy with this 22:20 <@anarcat> darthsteven: maybe we can talk later today? 22:21 <@anarcat> omega8cc: try cb0c2d2 22:21 <@darthsteven> sure, I'm around for the next 3 hours or so... 22:21 -!- scientist [~scientist@206.248.137.227] has joined #aegir 22:21 <@anarcat> ok 22:21 <+omega8cc> anarcat: sure, thanks 22:21 <@anarcat> i think that's it for our scrum unless somebody else wants to kick in... 22:21 <@anarcat> who's it? 22:22 <@anarcat> (who wants to upload the scrum log, that is) 22:22 <@mig5> i'll do it 22:22 <@darthsteven> I can 22:22 <@darthsteven> oh 22:22 <+omega8cc> :) 22:22 <@darthsteven> go for it mig5
Weekly Scrum IRC Log: 2011-07-26
[12:00pm] anarcat scrum time! [12:00pm] anarcat: hello people [12:00pm] univate: hello [12:00pm] anarcat: welcome to our morning weekly scrum [12:00pm] omega8cc: hi [12:00pm] anarcat: hey univate what's up [12:00pm] anarcat: alright [12:00pm] You left the chat by being disconnected from the server. [12:01pm] You rejoined the room. [12:01pm] You were granted voice by ChanServ. [12:01pm] anarcat: univate: i don't know! [12:01pm] anarcat: so our gsoc student is supposed to take care of that, but i didn't get any news from him since the evaluation [12:02pm] anarcat: which is really too bad [12:02pm] anarcat: i'll try to ping him by email now [12:02pm] mig5: anarcat: please send me an email before you go, re GSOC student [12:02pm] mig5: or cc me on that one ^^ [12:02pm] anarcat: alright, i'll do this right away [12:02pm] anarcat: other than that, i think that's it for me [12:03pm] mig5: nothing new from me, all my time atm is taken up by work and study [12:03pm] anarcat: alright, sounds like me except s/work and study/vacation/ [12:04pm] anarcat: mig5: @gmail is okay? [12:04pm] mig5: sure [12:04pm] omega8cc left the chat room. (Ping timeout: 240 seconds) [12:04pm] You are now known as omega8cc. [12:06pm] anarcat: okay, anyone else? [12:06pm] anarcat: darthsteven: ping [12:06pm] anarcat: omega8cc: how are the cron jobs? [12:07pm] darthsteven: anarcat: pong! [12:07pm] Irishgringo joined the chat room. [12:07pm] anarcat: ha! [12:07pm] anarcat: darthsteven: anything for the scrum? [12:07pm] anarcat: darthsteven: we need to coordinate for the training and the session [12:07pm] omega8cc: anarcat: it works great, but it should probably go in 1.3 no? as it is a critical for all users [12:07pm] darthsteven: nothing for the scrum [12:07pm] anarcat: darthsteven: ok [12:07pm] anarcat: darthsteven: i'm leaving the internets for a while on the 1st, so we would need to coordinate this week... email? [12:08pm] anarcat: omega8cc: well, not for *all*, the drush method works fine right? [12:08pm] darthsteven: anarcat: sure [12:08pm] omega8cc: yeah, so it is critical in 50% [12:08pm] anarcat: okay [12:08pm] anarcat: i don't think i'll be making a 1.3 release now [12:09pm] anarcat: if someone else wants to, be my guest! [12:09pm] omega8cc: as long as we have crush method as default, it is fine [12:09pm] omega8cc: drush [12:09pm] anarcat: univate: so the d7 port is handled by seth.vincent's sandboxes on drupal.org, if you want to followup on that it would be great [12:09pm] omega8cc: uh, Lion autocorrection is crazy [12:11pm] univate: anarcat: ok i will try and track down what is there [12:11pm] anarcat: univate: can you give me your mail addy again [12:14pm] univate: looks like seth.vincents last commits to his sandbox projects were 5 weeks ago [12:15pm] anarcat: quite possible [12:15pm] anarcat: he kinda stalled on the project at some point [12:15pm] anarcat: i'll push him again now [12:18pm] anarcat: so what's the next scrum time? [12:18pm] welly left the chat room. (Quit: Textual IRC Client: http://www.textualapp.com/) [12:18pm] anarcat: i won't be coordinating the next ones until october 1st [12:19pm] anarcat: so we need someone to take that over, or we suspend those for a while [12:19pm] mig5: I don't think we agreed on a new time, i'll coordinate with darthsteven / omega8cc [12:20pm] mig5: if we can't agree on a good time we'll suspend them or rotate people [12:20pm] anarcat: ok [12:20pm] mig5: or i'll move to europe [12:20pm] omega8cc: [12:21pm] omega8cc: it should be easy to find good time for EU/AU [12:23pm] mig5: yeah [12:24pm] jonhattan left the chat room. (Quit: llevaré una chaquete de guns'n'roses) [12:24pm] anarcat: there was a time proposal here: http://community.aegirproject.org/discuss/new-scrum-time-our-gsoc-student [12:28pm] omega8cc: mig5: what time works best for you? morning, evening? [12:36pm] Egyptian[Laptop] joined the chat room. [12:40pm] omega8cc: so maybe http://bit.ly/qGhjDr ? [12:40pm] mig5: evening usually better for me [12:41pm] omega8cc: 7, 8 or 9 pm? [12:41pm] mig5: 8 or 9 is fine, that link above is fine with me [12:41pm] mig5: let's see what darthsteven says [12:41pm] omega8cc: k [12:41pm] anarcat: mig5: ping [12:41pm] mig5: pong [12:42pm] q-rban left the chat room. (Remote host closed the connection) [12:42pm] q0rban joined the chat room. [12:43pm] anarcat: http://community.aegirproject.org/handbook/project-history [12:44pm] mig5: that's quite good! when did that appear [12:44pm] scientist joined the chat room. [12:45pm] anarcat: i just wrote it [12:45pm] mig5: nice [12:45pm] mig5: some of that would be good in the wikipedia page [12:46pm] mig5: (for those of you who missed it: http://en.wikipedia.org/wiki/Aegir_Hosting_System ) [12:46pm] anarcat: yeah, that's why i wrote it [12:46pm] mig5: [12:47pm] Irishgringo left the chat room. (Quit: ChatZilla 0.9.87 [Firefox 5.0.1/20110707182747]) [12:47pm] omega8cc: nice! [12:48pm] mig5: in other news, i rejoined twitter, and now feel dirty [12:49pm] mig5: and am studying for my OSCP, so if you find I discover any XSS or other holes in aegir, you know why http://www.offensive-security.com/online-information-security-training/p... [12:50pm] scottalan joined the chat room. [12:51pm] darthsteven: mig5, omega8cc: 10AM UTC is fine with me [12:52pm] omega8cc: mig5: I already grabbed that book about security thanks to your tweets, so it is a good thing (™) [12:54pm] omega8cc: darthsteven, mig5: so let's change the scrum time to http://bit.ly/qGhjDr then for the summer [12:54pm] darthsteven: agreed
Weekly Scrum IRC Log: 2011-08-02
20:00 <@mig5> ok, scrum time 20:00 <@mig5> as usual, no news from me this week :P 20:00 <@mig5> anarcat is away as we know 20:00 <@mig5> I have not heard from the GSOC student, so I will follow that up 20:01 <@mig5> there are currently no criticals in the queue which is nice, although I have vaguely heard about some issues with the cron queue, someone let me know if that's still an issue 20:01 <@mig5> aaaand I can't think of anything else significant, other than thanks to omega8cc for donating to hefring :) 20:02 <+omega8cc> haha 20:02 <@mig5> go for it darthsteven, omega8cc 20:02 <@darthsteven> I've done a little bit of work on the Aegir session for Drupalcon 20:02 <@darthsteven> hope to have that completed by the end of the week 20:02 <@mig5> oh nice 20:03 <@darthsteven> And then I hope to start writing docs again, but am away on holiday a lot over the next little while 20:03 <@darthsteven> but that's it from me 20:03 -!- aegir-jenkins [~PircBot@173-203-101-207.static.cloud-ips.com] has quit [Ping timeout: 252 seconds] 20:03 <@darthsteven> omega8cc? 20:04 <+omega8cc> I didn't submit any new patches, but I promise I will - related to the UI, soon 20:04 <+omega8cc> also, I released 1.0-boa-T-8.9, and it is using Aegir head 20:04 <+omega8cc> so good for receiving more feedback 20:05 <+omega8cc> and I noticed the sites cron is still an issue on some servers, when web based method is used 20:05 <+omega8cc> I will debug this and post more details soon 20:05 <@mig5> i am tired of this one in the queue: http://drupal.org/node/1111572 - I have been planning on removing a chunk of code from provision for that for a while, so I might do that in the 7.x branch and see how it goes 20:05 < hefring> http://drupal.org/node/1111572 => Undefined index: profiles deploy.provision.inc:84 => Provision, Code, normal, needs work, 14 comments, 2 IRC mentions 20:06 <@mig5> it is harmless but people keep carrying on about it 20:06 <@mig5> we have a fresh bug from ergonlogic too http://drupal.org/node/1236490 20:06 < hefring> http://drupal.org/node/1236490 => "View client" permission broken => Hostmaster (Aegir), Code, normal, active, 0 comments, 1 IRC mention 20:06 -!- aegir-jenkins [~PircBot@173-203-101-207.static.cloud-ips.com] has joined #aegir 20:07 <@mig5> aegir-jenkins returns.. that was weird 20:07 <@mig5> it's been up for over 100 days now, \o/ 20:07 <@darthsteven> mig5: 1111572, can we set up a test case that invokes that code, remove it and then see if anything breaks/changes 20:07 <+omega8cc> aegir-jenkins: welcome back :) 20:07 -aegir-jenkins:#aegir- omega8cc did you mean me? Unknown command 'welcome' 20:07 -aegir-jenkins:#aegir- Use '!78jujdd7yujufyyshhayuuufhhysjjhyew9i2 help' to get help! 20:07 <@mig5> hah 20:07 <+omega8cc> haha 20:08 <@mig5> oops, and there goes anarcat's clever idea to 'hide' the jenkins command... 20:08 <@darthsteven> hehe 20:08 <@darthsteven> that's genius! 20:08 <@darthsteven> anyone else have anything for the scrum? 20:09 <@mig5> darthsteven: that's the thing re: test case: we don't scan packages in the site-specific dir, which means those values are always null. anyway i promise to look at it 20:09 <@darthsteven> ok 20:09 < Aciid> I have something , never setup a VM with Amazon AWS 20:09 <@mig5> ok, scrum done, unless ergonlogic has anything to add 20:09 < Aciid> http://pastebin.com/pRt4UNeg - mail sent by Amazon 20:10 <+omega8cc> so, thanks for attending! 20:10 <@mig5> cheers
Weekly Scrum IRC Log: 2011-09-13
[20:15:26] <@darthsteven> Scrum time
[20:15:30] <@darthsteven> at last!
[20:15:49] <@mig5> yeah sorry about that
[20:15:52] <@mig5> i have neglected it a bit
[20:16:14] <@darthsteven> ping: darthsteven, mig5, ergonlogic, grugnog, mvc, omega8cc, sfyn, skwashd (time for a scrum!)
[20:16:23] <@darthsteven> mig5: I kept forgetting to have them!
[20:16:33] <@darthsteven> do you want to go first?
[20:16:52] <@darthsteven> mig5: do you want to go first?
[20:17:04] <@mig5> sure, that's easy: i've done nothing as usual
[20:17:06] <@mig5> :)
[20:17:09] <@omega8cc> lol
[20:17:13] <@mig5> sigh
[20:17:26] <@mig5> i want to look at getting those jenkins tasks into provision, but i haven't begun
[20:18:18] <@mig5> i have built some cool automatic deployments using jenkins + aegir, all tracking changes in git, which i want to screencast soon
[20:18:28] <@mig5> zero-touch deployment of dev > stage > live, good fun
[20:18:36] <@darthsteven> mig5: awesome!
[20:18:39] <@mig5> anyway, that's me..
[20:18:57] <@darthsteven> mig5: Thanks!
[20:19:08] <@darthsteven> omega8cc: anything from yourself?
[20:19:27] <@omega8cc> I already forgot what I did in the last few weeks :p but it was something, I'm sure! seriously, I probably helped with some drush issues and now working on a new theme, including mobile version for hostmaster
[20:19:46] <@mig5> ooer
[20:19:53] <@darthsteven> cool! (the theme stuff, not forgetting things)
[20:20:21] <@darthsteven> omega8cc: anything else?
[20:20:50] <@omega8cc> and the app for iOS and Android maybe is in the works, leveraging Aegir api where possible
[20:21:11] <@omega8cc> but it is an Apple-like secret! lol
[20:21:44] <@darthsteven> heh, well the scrum logs don't get published anywhere ;)
[20:21:56] <@omega8cc> but first, I will submit some simple UI related patches for the current theme
[20:22:02] <@omega8cc> haha
[20:22:26] <@omega8cc> I have some problems with it
[20:22:56] <@omega8cc> I don't want to include ctools and stuff just to override some blocks, for example
[20:23:12] <@omega8cc> so some patches will need to hit the core instead
[20:23:19] <@omega8cc> I mean hostmaster
[20:23:40] <@darthsteven> omega8cc: ok, well I reckon we might end up with ctools in 7.x-2.x before long anyway
[20:23:58] <@darthsteven> because I want views in there :)
[20:24:09] <@darthsteven> omega8cc: anything else?
[20:25:06] <@omega8cc> features_extra requires ctools, but we could too easily end up with fatcore instead of smallcore then
[20:25:08] * skwashd stands up
[20:25:46] <@omega8cc> so, features are nice, but always
[20:25:58] <@omega8cc> but not* always, err
[20:26:28] <@omega8cc> and that is it from me, I guess
[20:27:10] <@darthsteven> cool, thanks for the work on Aegir, as always
[20:27:17] <@darthsteven> skwashd: do you want to go next?
[20:27:45] <@skwashd> darthsteven: i have done 50% more work than mig5 on the same tasks as he has
[20:27:58] <@skwashd> darthsteven: and i plan to double my efforts on that task this week
[20:28:23] <@mig5> that sounds believable
[20:28:30] <@omega8cc> haha
[20:28:31] <@darthsteven> skwashd: sorry, what task was that?
[20:28:39] <@mig5> trolling
[20:28:43] <@skwashd> darthsteven: doing fuck all for aegir
[20:28:46] <@darthsteven> heh
[20:29:07] <@darthsteven> well, thanks for keeping us in the loop :)
[20:29:16] <@darthsteven> skwashd: anything else?
[20:29:22] <@skwashd> mig5: i said i'd only done 50% more than you ... if we're talking about trolling there would be another 0 in there :P
[20:29:41] <@darthsteven> moving on...
[20:29:45] <@darthsteven> I'll go next
[20:29:46] <@skwashd> darthsteven: no
[20:30:06] <@darthsteven> I fixed an issue with PHP 5.2 and Drupal 7.8
[20:30:50] <@darthsteven> I've asked about spinning the theme out of hostmaster, back into it's own project http://drupal.org/node/1269212
[20:30:51] <@hefring> http://drupal.org/node/1269212 => Spin Eldir back out to its own project => Hostmaster (Aegir), Code, normal, needs review, 0 comments, 1 IRC mention
[20:31:17] <@darthsteven> I built a 'harness' for testing Eldir for Drupal 7: http://drupal.org/sandbox/darthsteven/1267494
[20:31:37] <@darthsteven> I.e. it's a 'port' of Hostmaster's HTML output for D7
[20:32:09] <@darthsteven> finally I have ported Eldir to D7: https://github.com/darthsteven/eldir
[20:32:23] <@darthsteven> building on the work of Seth (SoC student)
[20:33:00] <@darthsteven> I think I've also fixed various little things in Aegir and tried to process the queues a little
[20:33:42] <@darthsteven> Future plans are to possibly make the 7.x-2.x branch actually use D7, and begin the porting...
[20:34:05] * beautifulmind has joined #aegir
[20:34:15] <@darthsteven> that's it from me
[20:34:22] <@darthsteven> anyone else?
[20:36:20] <@darthsteven> I'll give them a minute...
[20:37:46] <@darthsteven> </scrum>
[20:37:51] <@darthsteven> Thanks everyone!
Weekly Scrum IRC Log: 2011-09-20
20:01 <@darthsteven> scrum in 1 minute! 20:02 <@darthsteven> mig5, mvc, omega8cc, sfyn, skwashd; scrum in 1 min 20:03 <@darthsteven> scrum time! 20:03 <+omega8cc> hello! 20:03 <@darthsteven> scrum? 20:03 < hefring> scrum is Every Tuesday at 10h00 UTC: 03h00 San Francisco, USA (PST), 06h00 Montreal, Canada (EDT), 11h00 London, UK (CET), 20h00 Melbourne, Australia (EST) 20:03 <@darthsteven> indeed it is hefring 20:03 <@darthsteven> botsnack 20:03 < hefring> skynet thanks YOU! 20:03 <@darthsteven> who wants to go first? 20:04 <@mig5> just made it.. 20:04 <@darthsteven> mig5: Hello! 20:04 <@mig5> you go firsst, you have been on fire in the queue! 20:05 <@darthsteven> I fixed some issues that were in the queue, and worked on the site form a little 20:05 <@darthsteven> Hopefully we can now handle database passwords that include special characters 20:05 <@darthsteven> though we may be chasing that one around a few more dark corners yet 20:06 <@darthsteven> I'd like some feedback about the site form changes (which are in a dev branch) 20:06 <@darthsteven> the issue is... 20:06 <@darthsteven> http://drupal.org/node/1282280 20:06 < hefring> http://drupal.org/node/1282280 => Clean up the hosting form javascript => Hostmaster (Aegir), Code, normal, needs review, 2 comments, 1 IRC mention 20:06 <@darthsteven> I would really appreciate some reviews of that 20:07 <@mig5> i never understood any of that as i don't grok javascript - i'm glad you're tackling it 20:07 <@darthsteven> It's basically just adding an AJAX spinner, so you know that something is happening... 20:07 <@mig5> so i'm not sure how helpful i can be there but hopefully others.. 20:07 <@mig5> ah right 20:07 <@darthsteven> even just a test and 'I like the look of that' or 'Can we do this' would be helpful 20:08 <@darthsteven> Also 20:08 <@darthsteven> I installed Drupal 8 using Aegir last night, and it worked! 20:08 <@darthsteven> which isn't a major surprise, but it is fun 20:08 <@darthsteven> that's it from me 20:08 <@darthsteven> who wants to go next? 20:09 <@mig5> nothing from me, i see you want to add some drupal8 tests into jenkins, i can do that 20:09 <@mig5> still way too busy right now to look at moving tests into provision 20:09 <@mig5> hopefully more time early october when i go full-time consultant 20:10 <@darthsteven> mig5: do you have time for a quick chat after the scrum about Aegir stuff? 20:10 <@mig5> should do, it's either you or dinner, whatever happens first 20:10 <@mig5> :) 20:11 <@darthsteven> mig5: been thinking about the tests stuff, have some ideas and thoughts about it... 20:11 <@darthsteven> mig5: anything else to add? 20:11 <@mig5> nope, other than to say thanks to you specifically for working the issue queue like a nutter this week 20:11 <@darthsteven> mig5: thanks for coming to the scrum even if you don't have much to report! 20:11 <@mig5> keep it up but don't burn out :) 20:11 <@darthsteven> mig5: these are my normal levels of involvement in projects... :) 20:12 <@darthsteven> anyone else have anything for the scrum? 20:12 <+omega8cc> I don't have anything new related to the Aegir core this week to report, however I worked on some BOA new features and there is a chance I will submit some new patches for Nginx config (and finally) the menu/UI. 20:12 <+omega8cc> darthsteven: and also thanks for your work in the queue! 20:12 <@darthsteven> omega8cc: awesome! 20:13 <@mig5> omega8cc: i expect to hear you move into Linode Tokyo soon ;) 20:13 <@darthsteven> omega8cc: I think we should make tiding up the UI a priority for 2.x 20:13 <@mig5> sorry offtopic 20:13 <@mig5> apparently there was a drupal montreal on 20:13 <@darthsteven> anyone else? 20:13 <@mig5> i read some french tweets about it 20:13 <@mig5> something about an 'albert method', a guy from koumbit did a talk 20:14 <@mig5> i am keen to find out more, there are slides somewhere, search on twitter under #aegir 20:14 <@mig5> i think it's deployment-related, i'll see if i can find out more and get some english-language focus on it as it looked interesting 20:15 <@darthsteven> cool 20:15 <+omega8cc> http://t.co/gv7OisUe 20:15 <+omega8cc> or http://twitter.com/#!/omega8cc/status/114783126844223488 20:16 <@mig5> oh nice already in english 20:16 <+omega8cc> yep 20:16 <@mig5> again, soon, i will have a new screencast up 20:16 <@mig5> about my 'zero touch' deployment for aegir 20:16 <@mig5> using jenkins 20:16 <@mig5> that 'reacts' to git push in de/v/stage/live environments 20:16 <@darthsteven> mig5: cool, have you seen that someone is doing screencasts of Aegir stuff? 20:16 <@mig5> that will come 'soon' ;) 20:16 <@darthsteven> mig5: that would be awesome 20:16 <@mig5> darthsteven: yes i did, i watched one 20:17 <+omega8cc> mig5: cool! finally! :) 20:17 <@darthsteven> mig5: I've not watched them...so I've no idea if they're any good 20:17 <@darthsteven> To finish the scrum, I'd like to say... 20:18 <@darthsteven> that we seem to have had a bump in contributors, which is really cool! 20:18 <@mig5> yes! rock on, patchers! 20:18 <@darthsteven> There are also a few people who've started using Aegir, and are now blogging about the project 20:18 <@mig5> thanks GuyPaddock and halcyonCorsair 20:18 <@darthsteven> The 'Wedful' thing springs to mind 20:19 <@mig5> yes and hadsie, i think that was the first commercial 'payment' aegir system that implemented our api 20:19 <+omega8cc> right 20:19 <@darthsteven> So, with a bit of effort we make Aegir 2.x happen fast, and be super awesome in every way! 20:19 <@darthsteven> anyway 20:19 <@darthsteven> that's it for this scrum! 20:19 <@mig5> the pantheon project is getting a lot of publicity right now, i am keen to see people publish 'how to make your own pantheon' with aegir :) 20:19 <@mig5> just to balance things out :) 20:19 <@mig5> thanks scrummers 20:19 <@darthsteven> Thanks for attending 20:19 <+omega8cc> thanks 20:19 <@darthsteven> who wants to do the log thing? 20:20 <@mig5> i can 20:20 <@darthsteven> thanks!
Weekly Scrum IRC Log: 2011-09-27
[10:02am] darthsteven: mig5, ergonlogic, grugnog, mvc, omega8cc, sfyn: scrum time! [10:02am] omega8cc: hello! [10:02am] darthsteven: Welcome to our weekly Aegir Scrum [10:02am] darthsteven: Who would like to go first? [10:03am] merro left the chat room. [10:04am] darthsteven: omega8cc: would you like to start us off? [10:04am] beautifulmind joined the chat room. [10:05am] omega8cc: darthsteven: all my nginx related new stuff is a bit delayed as it needs more testing [10:05am] omega8cc: so I didn't submit any patches yet, but I hope to be able to do it this week [10:06am] omega8cc: also new boa release has been delayed for this reason [10:06am] darthsteven: okay [10:06am] darthsteven: well, looking forward to reviewing those bits [10:06am] merro joined the chat room. [10:07am] darthsteven: anyone else? [10:08am] darthsteven: I'll go next then [10:08am] darthsteven: I factored out the Jenkins tests to a drush command in provision [10:09am] darthsteven: so we can run them locally without python or Jenkins [10:09am] darthsteven: which is nice [10:09am] omega8cc: wow [10:09am] darthsteven: We can also have different tests for the different branches now too [10:10am] darthsteven: the tests are pretty basic, as we'll be using the Drush testing framework in a bit, when Drush 5 is out, but for now they're pretty basic [10:10am] darthsteven: also [10:10am] dfrt: /t [10:10am] darthsteven: I made the Jekins scripts use configuration from Jenkins, rather than new code in the repo [10:11am] darthsteven: so we can do releases much more easily [10:11am] darthsteven: speaking of which [10:11am] darthsteven: We release 1.4 [10:11am] darthsteven: *released [10:11am] darthsteven: Fixing a security issue [10:11am] darthsteven: Also [10:11am] darthsteven: I created the 6.x-2.x branch, which was where the 7.x-2.x branch was [10:11am] darthsteven: because... [10:12am] darthsteven: 7.x-2.x now contains Drupal 7 code! [10:12am] darthsteven: hurrah! [10:12am] darthsteven: Porting is underway [10:12am] omega8cc: [10:12am] darthsteven: I was going to let everyone know via the community site, but couldn't because that had issues with the OA upgrade that was forced upon it [10:12am] darthsteven: I will do that today though [10:14am] omega8cc: darthsteven: so, is our gSoC student work/idea already cancelled, I suppose? as you are doing it currently [10:14am] darthsteven: omega8cc: As far as I can tell, nothing really came of that [10:14am] omega8cc: yeah [10:15am] darthsteven: we can always get him porting one of the modules if that's not the case [10:15am] darthsteven: gSoC is over afaik anyway [10:15am] darthsteven: Anyone else want to say anything? [10:15am] omega8cc: so we may want to update our room header [10:20am] szczym left the chat room. (Read error: Operation timed out) [10:22am] omega8cc: darthsteven: Thanks for leading the work and our scrum! I can submit the log to c.a.o if you wish [10:22am] darthsteven: end of scrum!
Weekly Scrum IRC Log: 2011-10-18
[21:00:33] <@darthsteven> anarcat, darthsteven, mig5, mvc, sfyn, skwashd: Scrum time! [21:10:03] <@mig5> sorry, nothing to report [21:12:45] <@darthsteven> ok [21:12:47] <@darthsteven> thanks! [21:13:16] <@darthsteven> I applied some nasty patches to get the debian package installing correctly with drush 4.5 [21:13:46] <@darthsteven> I also got to the bottom of the problem when installing on debian, where it would consume all CPU and memory [21:13:52] <@darthsteven> which anarcat then fixed! [21:13:53] <@mig5> excellent. do we need to roll a new release for that drush issue? [21:14:57] <@darthsteven> I think we do [21:15:08] <@darthsteven> I was going to add an issue to the queue [21:15:20] <@darthsteven> See if we can do a release without anarcat :) [21:15:58] <@mig5> :) [21:20:50] <@darthsteven> anyone else? [21:21:06] <@darthsteven> speak now, or hold until next week... [21:21:29] <@darthsteven> oh, and I just posted about changing the times of these: http://community.aegirproject.org/discuss/change-scrum-time [21:36:06] <@darthsteven> okay [21:36:13] <@darthsteven> official end of scrum there
Weekly Scrum IRC Log: 2011-11-01
[21:00:57] Scrum time! [21:01:00] scrum? [21:01:00] scrum is Every Tuesday at 10h00 UTC: 03h00 San Francisco, USA (PST), 06h00 Montreal, Canada (EDT), 11h00 London, UK (CET), 20h00 Melbourne, Australia (EST) [21:01:08] go! [21:01:19] shall I go first? [21:01:26] okay, if you insist [21:01:40] Aegir 1.5 is out! [21:01:42] Hurrah [21:01:47] \o/ [21:01:57] Anarcat helped in advisory role [21:02:17] which means at least two people can complete the full release process, independently [21:02:26] excellent [21:02:50] and we found that Jenkins actually does most of the work, and we could get it to do most of the rest of Debian stuff very easily [21:02:57] other things: [21:03:10] I did some bug fixes [21:03:29] I completely re-factored the 6.x-2.x branch of provision to use autoloading for classes [21:03:41] so we can support contrib add-ons for provision [21:03:51] I still need to document those changes [21:04:01] will try to get a few mins to do that today [21:04:31] also, I increased the number of things tested by Jenkins, and improved the quality of the code running the tests a little [21:04:44] resulting in less code all round :) [21:05:05] Also, made the upgrade tests take parameters [21:05:09] which is handy [21:05:29] also we now test PHP 5.2, every 24 hours if there are changes [21:05:32] that's it from me [21:05:39] mig5 want to go next? [21:05:48] ok - incredible efforts as always darthsteven, thanks [21:05:52] that's amazing work [21:06:20] me - very little as always - i simply built a new debian squeeze image with backported php 5.2 from lenny, so that we could do those php 5.2 tests [21:06:57] i'm pondering giving an aegir talk at http://drupaldownunder.org in January 2012, only a couple monts away now, but i'd like to diverge away from the usual 'how to install aegir, basic usage' thing we always do, as a lot of people will have seen that [21:07:08] any suggestions or ideas, let me know. [21:07:11] that's about it from me [21:08:10] mig5: thanks as always [21:08:47] anyone else? [21:09:00] mig5: we can chat talks after the scrum? [21:09:23] sure [21:10:13] anyone else? I'll give you a minute! [21:11:31] okay [21:11:36] thanks for attending mig5 [21:11:37] i'll do log
Weekly Scrum IRC Log: 2011-11-14
07:01 <@mig5> well, scrum time :) 07:01 <@mig5> (I think!) 07:01 <@mig5> anarcat: ergonlogic: ^^ 07:01 <@mig5> I'll start. I've got myself back into the swing of things a bit, fixed some minor bugs, accidentally even committed some fixes by mistake that I wanted reviews of first, oh well :) 07:01 <@mig5> cleared out some old stuff in the queue that is irrelevant, are actually support requests, etc 07:02 <@mig5> submitted a talk for Drupal Down Under 07:02 <@mig5> http://drupaldownunder.org/session/aegir-skynet-drupal 07:02 <@mig5> I'm making some stickers and tshirts for that conference to give away and generally spruik Aegir 07:03 <@mig5> welcome back darthsteven 07:03 <@darthsteven> awesome stuff! 07:03 <@darthsteven> (my IRC reflector is doing funny things) 07:03 <@mig5> so, I'm hoping to continue to fix some small-ish bugs, help in the queue, etc 07:03 <@darthsteven> awesome stuff 07:03 <@mig5> i don't think i'll ever be a crazy big committer to the project since code is obviously not my biggest skill 07:04 <@mig5> but i'd like to get your help to make me contribute in some obvious way, even if it's just writing news/blog posts, managing the community portal etc 07:04 <@mig5> and perhaps be the '1.x' maintainer for basic bug fixes? 07:04 <@darthsteven> awesome, will try to push you :) 07:04 <@mig5> leaving you to concentrate on 2.x or something like that 07:04 <@mig5> these are just ideas anyway 07:05 <@mig5> anyway I think that's it for me. someone review my patches in the queue! :) 07:05 <@mig5> onto you darthsteven 07:05 <@anarcat> hey 07:05 <@darthsteven> ok 07:05 <@darthsteven> hey hey 07:05 <@anarcat> yay, i didnt miss the scrum :) 07:05 <@mig5> yay! 07:05 <@anarcat> go ahead darthsteven :) 07:06 <@darthsteven> I didn't do too much this week, I did some issue queue re-assignment 07:06 <@darthsteven> I set up some Jenkins jobs to monitor code style of hosting and provision using the Coder review module 07:06 <@darthsteven> see: http://ci.aegirproject.org/view/Code%20reviews/ 07:07 <@mig5> oh nice 07:07 <@darthsteven> not sure I'll have much time to work on stuff this week, but hoping to get back to 2.x asap 07:07 <@anarcat> nice 07:07 <@darthsteven> anarcat: over to you 07:08 <@anarcat> uh 07:08 <@anarcat> not much on my side either :) 07:08 <@anarcat> i am now a debian developer 07:08 <@mig5> \o/ 07:08 <@anarcat> and so i have pushed ergonlogic's drush_make package into debian 07:08 <@anarcat> my glorious achievements can be seen here: http://qa.debian.org/developer.php?login=anarcat 07:09 <@anarcat> i will try to start working again on 2.x this week, as my replacement is coming to an end 07:09 <@anarcat> but it's not easy 07:09 <@mig5> you are not allowed to quit :) 07:09 <@anarcat> i have worked on 1.5 release last week a little and found a critical bug in the debian package, which hosed a client server here 07:09 <@anarcat> yeah 07:09 <@anarcat> but it's all fine now 07:09 <@anarcat> and we discovered that most of the debian build stuff is now automated by jenkins, which is freaking cool 07:09 <@anarcat> and i support mig5 as a 1.x maintainer 07:10 <@mig5> ouch. so that critical bug only affected the debian package? 07:10 <@anarcat> yeah 07:10 <@mig5> do we need to reroll? lots of people use it now 07:10 <@anarcat> http://community.aegirproject.org/1.5#Known_issues 07:10 <@mig5> ah 07:10 <@mig5> :) 07:10 <@anarcat> aka https://drupal.org/node/1328316 07:10 <@anarcat> it's been rerolled already 07:10 <@anarcat> i wouldn't have let that one sit there :) 07:10 <@mig5> right right 07:10 <@mig5> :) 07:10 <@anarcat> so next steps for me 07:10 -!- darthsteven_ [~darthstev@109.123.86.70] has joined #aegir 07:10 -!- mode/#aegir [+o darthsteven_] by ChanServ 07:10 <@anarcat> backport drush_make, once it hits tseting 07:10 <@anarcat> testing* 07:11 <@anarcat> look into merging the daemon into 2.x, the sites/ directory magic and unit testing 07:11 <@anarcat> those impatient to see drush_make hit testing/backports can reload franatically this page: http://packages.qa.debian.org/d/drush-make.html 07:12 <@anarcat> oh, and we have submitted a Aegir training proposal for Denver 07:12 <@darthsteven> good stuff anarcat 07:12 <@anarcat> so i guess that means we'll go to denver :) 07:13 <@anarcat> well, i think that's it 07:13 <@mig5> ok, anyone else? 07:13 <@mig5> ergonlogic ^^ 07:13 <@anarcat> i think he's off 07:13 <@mig5> ok 07:13 <@anarcat> he was at drupalcamp toronto 07:13 <@anarcat> and is resting 07:13 <@mig5> oh that's right, i believe there was some aegir stuff happening there via him? 07:14 <@mig5> cool 07:14 <@anarcat> he presented aegir to some folks there and i heard through the grapevine that we may see derek again :) 07:14 <@mig5> who's derek? 07:14 <@anarcat> mig5: spiderman - worked on the dns stuff? 07:14 <@mig5> oh right! 07:14 <@mig5> I remember 07:15 <@mig5> ok, i'm off for breakfast. happy ot upload the log when i'm back in 20 min or so 07:15 <@mig5> thanks all! 07:15 <@anarcat> awesome 07:15 <@anarcat> yeah, i think that's it 07:16 <@darthsteven> thanks! 07:16 <@darthsteven> I think we need to do a 1.6 release for https://drupal.org/node/1330018 sooner rather than later 07:17 <@darthsteven> that bug is entirely my fault btw, major lack of testing on my part 07:17 <@anarcat> i see 07:17 <@anarcat> well, now you know how to make a release, maybe you can train mig5 to do it? :) 07:18 <@darthsteven> sure 07:29 <@darthsteven> I'll teach Jenkins how to do releases first
Weekly Scrum IRC Log: 2011-11-21
07:00 <@mig5> ok, time for a quick scrum 07:00 <@mig5> good morning 07:00 <@mig5> it appears darthsteven sends his apologies! 07:00 <@darthsteven> Nothing from me, will do a 1.6 release tomorrow if I have time! 07:00 <@mig5> so ping anarcat, ergonlogic, etc 07:00 <@mig5> oh he's here! :) 07:00 <@mig5> cool 07:00 <@mig5> thanks darthsteven 07:00 <@darthsteven> Now I'm gone... 07:01 <@mig5> not much from me this week: I posted the first of a fortnightly Aegir newsletter last wednesday, I got some good feedback from anarcat but that was about it :) 07:01 <@mig5> ok. 1.6 release will be tomorrow 07:01 <@mig5> my stickers have arrived, i ordered way too many for DrupalDownUnder, maybe I'll send some in the post to people :) 07:02 <@mig5> we fixed the Reset Password bug that broke resetting of passwords in Drupal 5, so that will go into the 1.6 release which is nice 07:02 <@mig5> that's all from me I think. anarcat? 07:03 < teeyza> so anyone have any ideas why my aegir wont upgrade? 07:03 <@mig5> right, i have a feeling anarcat isn't here, and we have a hijacker, so I might end the scrum earlier than usual.. 07:03 <@mig5> thanks all 07:14 <@anarcat> oops 07:14 <@anarcat> missed the scrum 07:14 <+ergonlogic> sorry 07:14 <@anarcat> mig5 / darthsteven : pong 07:14 <@mig5> hehe 07:14 <@anarcat> we were in our own planning session over here, maybe we can touchbase on that 07:14 <@mig5> sure thing 07:15 <@anarcat> we are reorienting my work a bit, away from http://community.aegirproject.org/discuss/back-vacation 07:15 < teeyza> my apologies mig i was corrected, so were stuck on step 4 of the that link 07:15 <@anarcat> we'll focus on the debian packaging and puppet manifests 07:15 <@anarcat> i'm going to work on a provision-upgrade command which will be basically like drush upgrade 07:15 <@anarcat> but really safe 07:15 <@anarcat> and designed specifically with the upgrade of aegir itself in mind 07:16 <@anarcat> so that we can ship the files with the aegir-hostmaster package 07:16 <@anarcat> and overwrite existing files, and run the update and so on 07:16 <@anarcat> we're also going to work on D7 LDAP integration so that Drupal can be used to manage LDAP groups and user memberships 07:16 <@mig5> will that open itself up to http://drupal.org/node/711746 in any way? 07:16 < hefring> http://drupal.org/node/711746 => allow upgrade of hostmaster sites from the frontend => Hosting, Code, minor, postponed, 3 comments, 1 IRC mention 07:16 <@anarcat> which is a bit offtopic here 07:16 <@anarcat> mig5: i don't know 07:16 <@anarcat> maybe 07:17 <@mig5> we need that external queue daemon :) 07:17 <@anarcat> so those are our priorities, shifting the other ones in the list down 07:17 <@anarcat> yes 07:17 <@anarcat> so that's still in the list 07:17 <@anarcat> but probably after christmas if not summer 07:17 <@mig5> sure 07:17 <@anarcat> same with unit tests, which get pushed away 07:17 <@anarcat> we're going to also sponsor more uc_hosting and hosting_services development 07:18 <@anarcat> with sfyn waiting to do a lot of improvements there, apparently 07:18 <@anarcat> expect the "official puppet module for aegir" to come out soon 07:18 <@anarcat> oh 07:18 <@mig5> awesome! :) 07:18 <@anarcat> and just random name-dropping here: openstack. 07:18 <@anarcat> that's it for me. 07:18 <@mig5> thanks anarcat 07:18 <+sfyn> 3~hey folks 07:19 <+sfyn> guess its scrum time 07:19 <@mig5> hey sfyn 07:19 <+ergonlogic> for my part, I'll be helping with the aegir puppet module 07:19 <+ergonlogic> yeah... a little late 07:19 < teeyza> hey mig, you have any other suggestions i could try? 07:19 <@anarcat> sfyn: it *was* :) 07:19 <@mig5> teeyza: please wait a bit, we are in the middle of a regular meeting 07:19 <+ergonlogic> and otherwise continuing to push aegir-up towards a release 07:19 <+sfyn> oops 07:19 < teeyza> kk 07:19 <@anarcat> sfyn: we had a great discussion on prioritizing work for aegir in the following week here 07:19 -!- mode/#aegir [+m] by anarcat 07:20 -!- mode/#aegir [+v mig5] by anarcat 07:20 -!- mode/#aegir [+v darthsteven] by anarcat, anarcat 07:20 <@mig5> thanks.. 07:20 <+ergonlogic> that's most of it for me 07:20 <+sfyn> short term my efforts are more on civicrm support right now 07:20 <@anarcat> sfyn: we'll be looking from feedback from you about estimates on how much time we should spend on hostnig_services integration this year 07:20 <@mig5> i'm very interested in the aegir puppet stuff, i use vagrant for dev now, let me know if you need extra eyes on that :) 07:20 <@anarcat> mig5: we certainly will! 07:20 <+ergonlogic> mig5: indeed, yes! 07:21 <@mig5> and i know puppet pretty well these days 07:21 <@anarcat> now that's cool 07:21 <+sfyn> anarcat: hosting_services is a mess right now 07:21 <@anarcat> sfyn: and yes, maybe we need to add provision_civi to our shopping list, if it's not budgeted in other teams 07:21 <@mig5> although ergonlogic taught me you can preseed directly in puppet, didn't know that :) 07:21 <+ergonlogic> :) 07:21 <@anarcat> sfyn: how much hours does that mean, "mess"? ;) 07:22 <+sfyn> anarcat: means i don't know 07:22 <@anarcat> sfyn: awesome 07:22 <@anarcat> sfyn: i've put a mostly random guess of 100h 07:22 <@anarcat> maybe we can try to fine-grain the thing in smaller iterations 07:22 <@anarcat> and do a better estimate 07:22 <@anarcat> but we don't need to do it now :) 07:22 <@anarcat> just a heads up that we'd like to push this out 07:22 <@anarcat> for the next reflexion meeting 07:22 <+sfyn> anarcat: ok 07:23 <@anarcat> alright, i think that's it for our part of the scrum, unless sfyn or ergonlogic have anything to add 07:23 <+sfyn> anarcat: I will break down the work for both projects over the week 07:23 <@anarcat> sfyn: awesome 07:23 -!- mode/#aegir [-m] by anarcat 07:23 <@mig5> thanks guys 07:23 <+ergonlogic> no, I'm done 07:23 <@anarcat> sorry for the censorship here :) 07:23 <+sfyn> thats it for me 07:23 <@anarcat> thanks\
Weekly Scrum IRC Log: 2011-11-28
[06:59:18] <mig5> scrum time in 1 minute i believe?
[06:59:25] <mig5> darthsteven: anarcat: ergonlogic:
[06:59:45] <anarcat> oh yeah!
[07:00:04] <mig5> ok!
[07:00:10] <mig5> scrum is now
[07:00:17] <anarcat> hi
[07:00:21] <mig5> i haven't done much this week at all :/
[07:00:29] * anarcat neither
[07:00:30] <mig5> and i just woke up so I can't remember if I did :)
[07:00:58] <mig5> well! if darthsteven doesn't show up, that's a very quick scrum! haha
[07:01:26] <mig5> I think ergonlogic worked on some new taxonomy/tagging/categorisation of discussion threads in the community site
[07:01:29] <mig5> i am yet to test it out
[07:01:40] <mig5> tomorrow i'll write my fortnightly aegir newsletter - do you think it should be monthly instead?
[07:02:04] <anarcat> yeah
[07:02:05] <anarcat> makes sense
[07:02:17] <anarcat> i was ill for two days last week, kinda knocked me off
[07:02:24] <mig5> the big news in it will be that 1.6 was released. speaking of that - we got a couple of good feedback on that release, and no big bug reports, so it must have been one of our most stable in a while
[07:02:30] <mig5> sorry to hear you were ill
[07:02:30] <anarcat> cool
[07:02:47] <anarcat> i did some debian work on my side, and the more i think about it, the more i wish to package hostmaster properly...
[07:02:58] <anarcat> i may work on that soon-ish
[07:03:04] <mig5> properly in what sense?
[07:03:10] <mig5> via debian official repos?
[07:03:15] <anarcat> properly as in, with the php files actually in the package
[07:03:16] <anarcat> yes
[07:03:32] <mig5> from memory we had some catch22 there
[07:03:35] <anarcat> but that's further down the road
[07:03:42] <anarcat> next step for me is ldap work
[07:03:43] <mig5> oh you mean with core inside it
[07:03:50] <anarcat> yeah
[07:03:53] <mig5> awesome
[07:04:04] <anarcat> basically, this would mean that we would drun drush_make during the build, not during the install
[07:06:08] <mig5> right right
[07:06:16] <anarcat> that's about it for me
[07:06:21] <mig5> thanks
[07:06:23] <anarcat> let's wait another 10 to see if our friends show up
[07:06:27] <mig5> yep
[07:06:44] <anarcat> oh! ergonlogic wasn't voiced!
[07:06:50] <mig5> hahaha
[07:06:51] <ergonlogic> aha
[07:06:56] <mig5> geeeez us tyrants
[07:07:03] <ergonlogic> Hi all, sorry I'm late
[07:07:11] <mig5> hi!
[07:07:11] <hefring> privet
[07:07:20] <anarcat> hum
[07:07:41] <mig5> oh my site, and hefring, are now managed by Jenkins :) whaaaaat?
[07:07:52] <anarcat> ergonlogic: you do not seem to be identified with nickserv, which is why you are not voiced
[07:07:52] <ergonlogic> so, yeah I added a simpler tagging mechanism on c.a.o
[07:08:33] <ergonlogic> I normally am...
[07:08:34] <ergonlogic> anyway, you can now tag without having to edit the node
[07:08:47] <ergonlogic> and we can filter discussions by keyword
[07:09:17] <mig5> sweet *me goes and filters out all mention of centos/redhat* :)
[07:09:38] <ergonlogic> this was mostly in response to a suggestion by AquaticDisorder to have per-OS discussions
[07:09:46] <ergonlogic> exactly :)
[07:10:06] <mig5> hmm i search for 'centos' and i get no results
[07:10:13] <mig5> same with 'debian'
[07:10:25] <ergonlogic> are there any nodes tagged as such?
[07:10:30] <mig5> oh duh
[07:10:32] <mig5> probably not! :)
[07:10:49] <ergonlogic> it's using the 'keywords' vocabulary
[07:10:55] <mig5> does it only search discussion threads?
[07:10:56] <mig5> gotcha
[07:11:00] <ergonlogic> which previously hadn;t been available for discussions
[07:11:04] <mig5> so it's probably that we have few tagged nodes
[07:11:04] <mig5> yep
[07:11:10] <ergonlogic> for now, just discussions
[07:11:17] <mig5> sorry still trying to wake up :)
[07:11:22] <ergonlogic> :)
[07:12:00] <anarcat> hey this reminds me - the twitter block seems to be broken
[07:12:07] <mig5> yeah was looking at that the other day
[07:12:08] <anarcat> last post is from may or something
[07:12:09] <mig5> i can't work it out
[07:12:24] <mig5> was looking at the view, but i think it's the feeds system itself... the feed URL is correct
[07:12:32] <ergonlogic> and I'm finally re-focusing on getting openatria.com running as a reference implemnetation of the Aegir-SaaS stack stuff I'd been working on
[07:12:33] <mig5> i think it's just not pulling in new items properly
[07:12:50] <anarcat> that sounds like a reasonable assumption
[07:12:54] <ergonlogic> and expect an alpha release of aegir-up this week
[07:13:17] <ergonlogic> I can take a look at that
[07:14:34] <ergonlogic> re being identified... I switched to using a proxy, could that exaplain it?
[07:14:36] <mig5> maybe something for after the scrum: i noticed we can't install aegir on a non-standard port. eg port 8080 (for vagrant direct 8080 to 8080 mapping)
[07:14:57] <mig5> we totally ignore --http_port in hostmaster-install. not sure why, it *ought* to be trivial surely? (famous last words)
[07:16:42] <ergonlogic> mig5: re vagrant, at one point I had aegir set up with dns, and then just added my vagrantbox as a dns server in resolv.conf
[07:18:32] <mig5> ah so you got arund the login links being port 80 etc by resolving them back to the vagrant machine itself, and the port 80 then NAT to the 8080 on the vagrant box too
[07:18:49] <mig5> i stil lreckon we should support non-standard port on fresh install: what if I want to put varnish in front of my aegir box for example
[07:18:54] <mig5> sure i can fiddle about after the install but yeah
[07:19:01] <mig5> anyway - i think we are done for scrum
[07:19:04] <ergonlogic> agreed
[07:19:07] <mig5> darthsteven appears MIA today
[07:19:13] <mig5> thanks all
Weekly Scrum IRC Log: 2011-12-12
[07:01:33] scrum time - any devs here? anarcat: ergonlogic: [07:01:38] hey [07:01:38] eh oh [07:01:39] yes [07:01:41] I believe darthsteven is currently at a Drupal meetup [07:01:44] hello! [07:01:44] ni hao [07:01:49] very little for me again [07:01:57] i have released this nifty module that waglo wrote: http://drupal.org/project/dblogin [07:02:16] ooh interesting [07:02:51] Hello! [07:02:51] yo [07:02:56] very little from me too. i sent out a silly newsletter last wednesday to be in timing for Vertice's 1 year anniversary [07:02:59] hey darthsteven, you made it! :) [07:03:03] Nothing from me today, hoping to do some real work on Aegir soon though! [07:03:09] hey darthsteven [07:03:18] darthsteven: i was wondering - how is the d7 port? [07:03:19] mig5: Thanks for that [07:03:32] Hah, stalled [07:03:45] I have 0 time to spend on aegir at the moment [07:03:49] i see [07:03:52] i know your pain [07:03:53] That will change in Jan [07:03:58] cool [07:04:12] i note that ohloh thinks we have decreasing overtime development [07:04:23] probably true [07:04:26] I'm trying to convince my employer to sponsor some of my time on Aegir [07:05:01] Pretty much will be able to spend 1 day every three weeks [07:05:18] Sorry, I'll be quiet now. [07:05:30] ok [07:05:36] pretty much the same situation for me [07:05:55] so basically, we're looking for sponsors and workforce :) [07:06:38] Yup [07:07:08] I want to donate money to the project, can someone let me know how :) [07:07:43] don't we have a paypal account? :) [07:07:59] i think koumbit do ;) [07:08:14] ah right, that's what that button does [07:08:31] i wonder how many donatiosn we receive there [07:08:36] me too :P [07:08:37] * anarcat will take a look [07:10:08] the answer is 0$ [07:10:15] whee [07:10:28] i thought i recalled ergonlogic or someone thanking some people in some release notes [07:10:44] interestingly enough, 6 of those are "incomplete transaction" [07:10:59] people donating between 18 and 100$ have canceled their submission, apparently [07:11:10] yeah 'http://community.aegirproject.org/1.3' [07:11:11] oh ok [07:11:19] weird [07:11:35] yeah [07:11:37] i wonder [07:11:48] i see my failed donation there, and it happened after i hit cancel in paypal [07:11:54] i'll try to do a real donation to see if it works [07:13:20] me too [07:13:21] this shit doesn't work [07:13:28] seemed to work for me [07:13:29] dude, did you just donate a lot? [07:13:34] $50 [07:13:43] yeah [07:13:47] mig5: it says: En attente (Incomplete Transaction) [07:13:49] that's a lot? [07:13:50] hmm [07:13:53] that's no good [07:13:54] well, for a test :) [07:14:18] i got a transaction id but no confirmation email [07:14:45] it doesn't show up in my paypal activity though [07:14:50] so probably just didn't do anything [07:15:15] civicrm, i love you [07:15:26] so that button freaking doesn't work [07:15:40] ergonlogic: the donate button in the community site doesn't work [07:15:57] we should just create a paypal account for the project and be done with it, remove the koumbit intermediary [07:16:09] i think we'll need more dev hands on deck.. let's be honest.. devseed left us, and many others, in the lurch, and i don't think we ever fully recovered [07:16:12] mig5 / darthsteven : ... and then us three can share the access in some way [07:16:22] i wish i could code a bit better or i'd do it, but i don't think it's my strength [07:16:38] i agree we need more people [07:16:45] i will not comment on your skills, you never listen to me :P [07:17:00] will i git log | grep migression to make my point? :) [07:17:22] anyways. next month i am at Drupal DownUnder talking aegir, i'll try and evangelise and drum up dev interest [07:17:29] oh! how was Drupalcamp NYC, how did the session go? [07:17:43] the session went well! [07:17:56] i got to talk with the pantheon guys about how they do intersite security, it was really interesting [07:18:10] ah nice [07:18:20] are they in fact doing multisite? [07:18:40] yeah, they switched away from one vserver per site [07:18:48] interesting [07:19:28] not sure the context, but seeing pantheon and aegir in the same chat sounds exciteing [07:19:54] it's good to trade war stories :) [07:20:13] hehe [07:20:20] yeah, it was great [07:20:33] what amazes me about pantheon is the deliberate simplicity of it.. not too many features, quite spartan in its approach [07:20:47] you have to be good to avoid feature creep like that [07:20:58] anyways - shall we call that a scrum end? [07:21:13] yes, i think we're done
Weekly Scrum IRC Log: 2012-01-09
14:56 mig5@> shall we do a scrum darthsteven anarcat ergonlogic omega8cc 15:02 anarcat@> oh yes! 15:02 anarcat@> scrum time! 15:03 omega8cc+> hi! 15:03 hefring > privet 15:03 anarcat@> alright 15:04 anarcat@> well, for myself i am just back from vacation 15:04 anarcat@> still swamped with a lot of non-aegir related work 15:04 anarcat@> but hopefully i will get to fix some pesky issues that have been bothering our clients and workers here 15:05 anarcat@> one low hanging fruit we have is to deploy those module automatically on new sites: http://drupal.org/node/1157114 15:05 hefring > http://drupal.org/node/1157114 => DB Login => 0 comments, 1 IRC mention 15:06 (*) anarcat looking for the second one 15:06 anarcat@> http://drupal.org/project/environment_indicator 15:06 anarcat@> there 15:06 anarcat@> the other things are: exportable backups and the pesky files directory 15:06 anarcat@> i am looking for ways to optimise the migrations 15:06 anarcat@> because right now we end up tarring and untarring a lot of the same crap every week 15:07 anarcat@> and i'm looking for ways to optimise that 15:07 anarcat@> so suggestions welcome there 15:07 anarcat@> the exportable backups is just to have a way to hardlink the backup in the @hostmaster/files directory with a hard-to-guess filename so that only the user can download it 15:11 mig5@> interesting 15:11 mig5@> sorry, dealing with a nagios alert here at the same time 15:11 anarcat@> fun :) 15:11 anarcat@> i think that's pretty much it for me 15:12 mig5@> not much from me this week.. need to find some more time to look at http://drupal.org/node/1388906 again. would be nice to test the nginx stuff more, not that we don't trust omega8cc of course :) 15:12 hefring > http://drupal.org/node/1388906 => It is possible to break Nginx based install with upload progress by enabling nginx_ssl feature => Provision, Code, critical, needs review, 5 comments, 1 IRC mention 15:12 mig5@> i'm otherwise just swamped with work, it's insane 15:12 anarcat@> i know how you feel 15:12 anarcat@> my return from vacation is insane 15:12 mig5@> i give my aegir talk at Druapl DownUnder on saturday, should be good 15:12 mig5@> how was NYC ? 15:12 anarcat@> we had a compromised ftp account that sent out 30k spams on my first day :P 15:12 mig5@> ugh! 15:12 anarcat@> it was good 15:12 anarcat@> it was only one day! 15:12 anarcat@> but i did a aegir pres with ergonlogic 15:12 anarcat@> and i think we got a few people in 15:13 anarcat@> i met with the pantheon guys, and they have a *great* product 15:13 anarcat@> not open source of course :P 15:13 mig5@> yeah. it's nice and simple 15:13 anarcat@> but they really got somethign going with their dev/staging/prod thingy 15:13 mig5@> yes and non-free :( 15:13 ergonlogic > hi 15:13 anarcat@> well, i got invite codes 15:13 anarcat@> i promised josh_k_ i would try to crack their thing 15:13 mig5@> haha 15:13 anarcat@> so far didn't find the time 15:13 anarcat@> but they really have good security tools 15:14 anarcat@> they are using cgroups, fcgi (or similar) 15:14 mig5@> David Strauss knows his stuff 15:14 anarcat@> so basically, you are renting out process groups 15:14 mig5@> and that valhalla filesystem he's using for clustering and HA, sounds interesting 15:14 anarcat@> as opposed to whole vservers, which was the old way 15:14 anarcat@> yeah that's nice too 15:14 mig5@> what are they using underneath: debian? 15:14 mig5@> or redhat? 15:14 anarcat@> i didn't troll, so i don't know ;) 15:14 mig5@> i seem to recall a redhat feature that rented processes 15:15 anarcat@> well, cgroups allow for fine-grained process isolation 15:15 anarcat@> cgroups is the kernel option that allows vservers (or linux containers as the call them now) to work 15:15 mig5@> yeh 15:15 anarcat@> they also use selinux 15:15 anarcat@> they seem really into it 15:15 anarcat@> pretty cool 15:15 anarcat@> anyways 15:15 mig5@> i'm surprised they don't just build a machine per client or something.. it's not that expensive these days 15:15 anarcat@> enough with pantheon :P 15:15 mig5@> yeah :) 15:15 anarcat@> mig5: that's what they were doing before 15:16 anarcat@> i think it was *more* expensive than clustering in a single box 15:16 anarcat@> you got to get some edge in some place 15:16 anarcat@> so anyways 15:16 anarcat@> ergonlogic hi! 15:16 anarcat@> ergonlogic: something to add to our first scrum of the year? 15:16 anarcat@> (or is it?) 15:16 anarcat@> :) 15:16 ergonlogic > well, I'm just back from vacation too 15:16 anarcat@> welcome back :) 15:17 ergonlogic > so trying to get back into the swing of things 15:17 ergonlogic > I'm working out the kinks to start doing to Aegir 101 screencasts 15:17 ergonlogic > s/to/some/g 15:17 ergonlogic > and fighting with vimeo... they don;t like .ogv apparently 15:18 anarcat@> them basterds :) 15:18 ergonlogic > but anyway, just "how to create a site with Aegir" kinda stuff 15:18 ergonlogic > introductory for new clients, at this point 15:19 ergonlogic > other than that, just before the holidays, anarcat and I published a couple puppet modules 15:19 ergonlogic > one for drush and another for aegir 15:19 ergonlogic > on d.o 15:19 anarcat@> oh yeah we did! 15:19 anarcat@> actually *you* did :) 15:19 ergonlogic > well, I published them, sure... 15:19 anarcat@> but yeah, we polished a aegir module 15:19 ergonlogic > but *we* worked on getting them publishable 15:19 anarcat@> unfortunately it still relies on debian 15:19 anarcat@> ie. it doesn't do the manual install if you're *not* on debian 15:19 mig5@> oh, exciting! where are the modules? 15:19 anarcat@> which would have been awesome 15:20 ergonlogic > I posted a dozen or so issues on them for cleanup 15:20 anarcat@> well, it *should* be on http://community.aegirproject.org/contrib-modules 15:20 anarcat@> but it isn't :P 15:20 mig5@> i'll work this into a new aegir newsletter :) 15:20 ergonlogic > I'll get the urls 15:20 anarcat@> and i'll add them to the page 15:20 ergonlogic > http://drupal.org/project/puppet-aegir 15:20 ergonlogic > http://drupal.org/project/puppet-drush 15:21 mig5@> i love this: server goes down, hosting company's support system and all their nameservers go down as well. excellent 15:21 mig5@> if it wasn't for DNS cache they'd be uncontactable to tell them about it 15:21 (*) anarcat notices the twitter block on c.a.o is still broken 15:21 ergonlogic > they depend on a couple other puppet modules, one from Koumbit, and another from RiseUp, which is documented on the project pages 15:21 anarcat@> mig5: dns -> ouch 15:21 ergonlogic > sigh... 15:21 mig5@> great stuff re: puppet 15:22 mig5@> looking forward to trying that 15:22 ergonlogic > anyway, I also updated Aegir Up to use the new puppet modules 15:22 ergonlogic > http://drupal.org/project/aegir-up 15:22 ergonlogic > using git subtrees 15:22 anarcat@> mig5: another thing for your newsletter: arch linux support! http://community.aegirproject.org/node/389/revisions/view/2683/2684 15:23 mig5@> yes! saw that 15:23 ergonlogic > so I now have about a dozen branches to wrangle 15:23 mig5@> and did you see darthsteven's crazy article where he mass-migrated sites between two aegirs 15:23 ergonlogic > but seems more maintainable than sub-modules 15:23 ergonlogic > indeed yes! 15:23 ergonlogic > I want to try it out this week or next 15:24 ergonlogic > the mass migration stuff looks very cool 15:24 ergonlogic > it's also the initial reason I started aegir-up, to test that sort of thing 15:25 anarcat@> http://community.aegirproject.org/contrib-modules#Puppet_modules 15:25 ergonlogic > anarcat: thanks! 15:26 ergonlogic > I think that's it from me... 15:26 anarcat@> ok, i think we're done then 15:27 omega8cc+> I'm also back at work and plan to port some never submitted patches to 1.x, as we (boa) are using already 2.x code so it was a bit out of sync between boa and official Nginx support in Aegir 1.x 15:27 ergonlogic > I'd really appreciate some feedback on Aegir-up, btw 15:27 anarcat@> unless darthsteven magically flys in 15:27 anarcat@> with his cape 15:27 anarcat@> oh, omega8cc ! hi! :) 15:27 omega8cc+> plus, I promised to submit standalone Aegir on Nginx how-to (step by step). We also tested and helped to fix a few distros so they work with Aegir and this list grows nicely :) 15:27 omega8cc+> hi anarcat! and all 15:27 anarcat@> nice! 15:27 anarcat@> omega8cc: any work on a debian package for nginx? :) 15:28 omega8cc+> anarcat: there is chance it will be soon! on the table already 15:28 omega8cc+> as we are moving with everything to things like deb, puppet etc 15:28 darthsteve@> Sorry I missed the scrum! 15:28 anarcat@> nice! 15:28 darthsteve@> In another meeting 15:28 anarcat@> i am really happy to hear that! 15:29 anarcat@> omega8cc: if you need help with the package, i already have ideas on how to do it 15:29 darthsteve@> Nothing really to add though 15:29 ergonlogic > omega8cc: I'd be happy to integrate nginx patches to the puppet-aegir module :) 15:29 ergonlogic > hi steve 15:29 anarcat@> omega8cc: basically, i think that it can be done without creating a new package - just add "| nginx" as a dependency and make a debconf dialog (or autodetection) to figure out which to setup 15:29 ergonlogic > steven 15:29 omega8cc+> anarcat: I will ask for your assistance for sure, thanks! 15:29 darthsteve@> Hoping to get back to working on Aegir within the next few weeks. 15:29 darthsteve@> Sorry for interrupting! 15:30 omega8cc+> anarcat: exactly, it shouldn't be a separate package 15:30 ergonlogic > darthsteven: thanks for the mass migration write-up 15:30 anarcat@> cool 15:30 ergonlogic > darthsteven: I'll be trying it out soon 15:30 omega8cc+> ergonlogic: sure, I will let you know 15:31 darthsteve@> ergonlogic: Yeah, no doubt I removed a vital part of it while writing it up. Pester me of that's the case 15:31 ergonlogic > darthsteven: you can bet on it ;) 15:32 darthsteve@> Some of it will be in Aegir core at some point, so I'd rather find the bugs now :) 15:33 omega8cc+> darthsteven: hi, in the meantime we (boa) are testing the 2.x code in production :) 15:34 omega8cc+> so far, absolutely no issues 15:34 darthsteve@> omega8cc: Hehe, just you wait... 15:35 omega8cc+> ;) 15:35 darthsteve@> omega8cc: Though hopefully we can get some more tests written 15:40 ergonlogic > ok then... so, I'll assume that's the end of the scrum and post the log to c.a.o 15:40 mig5@> thanks ergonlogic
Weekly Scrum IRC Log: 2012-02-06
14:59 darthsteve@> scrum in a couple of mins 15:02 darthsteve@> mig5, anarcat, EclipseGc, ergonlogic, hadsie, mvc, omega8cc, sfyn, skwashd: scrum! 15:04 anarcat@> yay! 15:04 anarcat@> i'm here for once 15:04 ergonlogic+> hi there 15:04 anarcat@> hey ergonlogic 15:04 omega8cc+> hi 15:04 hadsie+> hey 15:04 hefring > eh oh 15:05 anarcat@> alright, should i start? i have lots to say :) 15:05 anarcat@> so i'll just go ahead 15:05 ergonlogic+> sure 15:05 anarcat@> alright 15:05 anarcat@> i wrote this crazy web_pack module last week 15:05 anarcat@> in basically 3 hours, i fixed most of the issues i was having with our multi-server support, or at least the "cluster" use case 15:06 anarcat@> i have posted this article requesting comments for total replacement of the current cluster module with the new implementation: 15:06 anarcat@> http://community.aegirproject.org/discuss/are-people-using-cluster-modul... 15:06 anarcat@> s/#comment.*// 15:06 anarcat@> anyways 15:06 darthsteve@> cool 15:06 anarcat@> the idea is to stop syncing the damn platforms and sites all over the place all the time 15:06 anarcat@> we only sync the ~aegir/config directory and reload apache 15:06 darthsteve@> replace replace replace 15:07 anarcat@> we assume /var/aegir/platforms is NFS-mounted 15:07 anarcat@> which brings me to: i think we should install the hostmaster platform in /var/aegir/platforms 15:07 anarcat@> it's inconsistent to have it where it is, and i don't see what purpose it serves 15:07 anarcat@> + it makes sharing "all the platforms" harder 15:07 anarcat@> food for thought 15:07 anarcat@> i have also worked on the provisionacl module 15:07 anarcat@> to make the sites directory accessible to the client group 15:08 anarcat@> but there are a few snafus in there that need working out 15:08 anarcat@> basically: http://drupal.org/node/1428526 15:08 hefring > http://drupal.org/node/1428526 => Files created by www-data user => Provision ACL support, Code, normal, active, 0 comments, 1 IRC mention 15:08 anarcat@> and http://drupal.org/node/1416056 15:08 hefring > http://drupal.org/node/1416056 => give access to entire site directory (was: add support for .git) => Provision ACL support, Code, normal, active, 8 comments, 1 IRC mention 15:08 anarcat@> otherwise i am trying to think more about how to optimise the migrate task so that it doesn't tar and untar all those damn files all the time 15:09 anarcat@> so far i came up with the idea that files/ should be a symlink to a version-controlled storage 15:09 anarcat@> and that therefore shouldn't eb backed up 15:09 anarcat@> so backups would now mean "backup the database", not "backup the filesystem" 15:09 anarcat@> that would be a plugin, i guess 15:09 anarcat@> i had a bit more time to work on aegir these days, which feels good 15:10 hadsie+> that would be awesome :) 15:10 darthsteve@> hmm…interesting 15:10 anarcat@> especially since it's client work paying for that 15:10 anarcat@> looking for more of those opportunities here 15:10 anarcat@> oh and i had issues... with 2.x 15:10 anarcat@> i wrote the pack module for 2.x, but since we don't deploy 2.x anywhere, i had to backport it to 1.x, which means now i have 4 branches (2 per module, 2 per version) for this simple module 15:11 anarcat@> kind of stupid 15:11 darthsteve@> ouch 15:11 anarcat@> i also wonder if that module could just be contrib, i didn't bother 15:11 anarcat@> i think that's it for me 15:11 darthsteve@> well looks like you got up to lots of stuff 15:11 darthsteve@> thanks! 15:11 anarcat@> :) 15:11 darthsteve@> anyone else want to go next? 15:12 ergonlogic+> you? 15:12 ergonlogic+> ok, me then 15:13 darthsteve@> ergonlogic: you go 15:13 ergonlogic+> I've been mostly working on Open Atrium features these past couple weeks 15:13 ergonlogic+> and not enough on Aegir 15:13 darthsteve@> tut tut 15:13 ergonlogic+> but that should change soon, as I gear up to launch openatria.com 15:13 darthsteve@> oh cool 15:13 ergonlogic+> I'll basically be solidifying the contrib modules I've written 15:14 ergonlogic+> but that ususally involves some patches to aegir too 15:14 ergonlogic+> I also nee to update aegir-up, as an update to vagrant has now broken it 15:14 ergonlogic+> and I realized that at least 2 people other than myself were using it 15:14 ergonlogic+> so that'll be a priority this week 15:15 ergonlogic+> and I'll be reworking our puppet modules 15:15 ergonlogic+> mostly to parameterize the classes 15:15 darthsteve@> awesome 15:15 (*) anarcat notes that vagrant has now hit debian unstable, which means he may start using it... some day :P 15:15 darthsteve@> need to talk to you about those puppet modules... 15:15 ergonlogic+> so we can point to other db servers, etc. 15:15 anarcat@> darthsteven: can do 15:15 ergonlogic+> darthsteven: whenever you'd like 15:16 anarcat@> snap :) 15:16 ergonlogic+> anyway, that's my plan for the coming weeks 15:16 darthsteve@> cool, thanks also 15:16 anarcat@> alright, we're splurging out of our timebox here 15:16 anarcat@> anyone else? 15:16 ergonlogic+> oh, and I'd like to get back to screencasting, but that'll probably have to wait 15:16 darthsteve@> me! 15:16 anarcat@> go go ! :) 15:16 darthsteve@> I've been working on something called 'Pergola' 15:16 darthsteve@> which is an open source version of the Pantheon server setup scripts 15:17 darthsteve@> using puppet 15:17 darthsteve@> so not much Aegir time 15:17 anarcat@> whoa 15:17 darthsteve@> but, been trying to get back into Aegir stuff 15:17 anarcat@> "A pergola, arbor or arbour is a garden feature forming a shaded walkway, passageway or sitting area of vertical posts or pillars that usually support cross-beams and a sturdy open lattice, often upon which woody vines are trained. As a type of gazebo, it may also be an extension of a building, or serve as protection for an open terrace or a link between pavilions." 15:17 anarcat@> - wikipedia 15:17 darthsteve@> and think about the D7 port and 6.x-2.x etc. 15:17 anarcat@> pergola isn't aegir driven? 15:18 darthsteve@> also, I did a migration the other day that failed in a really horrible way 15:18 darthsteve@> which was interesting 15:18 anarcat@> indeed 15:18 anarcat@> horrible as in killing kittens? 15:18 darthsteve@> Will talk about Pergola after the scrum, mentioned it, because that's where I've been spending my time :) 15:18 anarcat@> ok 15:18 darthsteve@> horrible as in the apache config got rolled back, but the site's files did not 15:19 anarcat@> we should talk about [67].x-2.x after too 15:19 anarcat@> ouch 15:19 darthsteve@> I had a look at Pantheon and dev cloud for inspiration for getting Aegir easier to use 15:19 darthsteve@> seeing how/if they handle 'platforms' and 'sites' 15:19 anarcat@> yeah, they really have their shit together 15:20 darthsteve@> but that's it from me 15:20 darthsteve@> anyone else? 15:20 anarcat@> hadsie ? omega8cc ? 15:20 anarcat@> don't be shy, we'd love to hear you talk about your aegir work! :) 15:20 omega8cc+> I have a ton of patches collected for nginx config, ready to submit, but who will have a time to review them? mig5 maybe? :) 15:20 omega8cc+> Anyway, it is a first step to fully (finally) sync my already fork with upstream before I will be able to seriously get the .deb thing done for nginx folks right, which is the next step to get done. 15:21 anarcat@> if they're marked as needs review, i may just do that, because i'm considering a 1.7 release with the pack module :) 15:21 anarcat@> nice, we should talk 15:21 omega8cc+> ok, expect a flood there :) 15:21 omega8cc+> And that is it from me, I guess. 15:21 anarcat@> alright 15:21 anarcat@> so unless hadsie or anyone else has something to add, i guess we can close this scrum 15:21 anarcat@> and thank you everyone for attending!
Weekly Scrum IRC Log: 2012-02-13
[06:59:17] <anarcat> scrum?
[06:59:17] <hefring> Every Monday at 20h00 UTC: 12h00 San Francisco, USA (PDT), 15h00 Montreal, Canada (EDT), 20h00 London, UK (GMT), 07h00 (Tuesday) Melbourne, Australia (EST)
[07:00:10] <ergonlogic> hi all
[07:00:46] <darthsteven> hi all!
[07:00:54] <anarcat> hi
[07:00:54] <hefring> what's up
[07:01:22] <anarcat> so i can start i guess
[07:01:30] <anarcat> i have worked a bit more on aegir in the last week
[07:01:40] <anarcat> i have merged the pack branch into 1.x and 2.x
[07:01:47] <anarcat> assuming that it was okay to ship 1.7 with it
[07:01:59] <anarcat> i am working on the import documentation today
[07:02:11] <anarcat> what else
[07:02:19] <anarcat> oh, i want to merge the debian branch into 1.x and 2.x
[07:02:23] <anarcat> so that we can start building 2.x packages
[07:02:38] <anarcat> we need to figure out *where* they would be uploaded, because that would override the 1.x packages
[07:02:46] <anarcat> but maybe we can just make another reprepro archive or something
[07:03:06] <anarcat> what else what else
[07:03:22] <anarcat> oh
[07:03:31] <anarcat> i would really like to see us switch to vagrant for jenkins
[07:03:35] <anarcat> but i have no time to take care of it
[07:03:45] <anarcat> i am tired of false positives and i'd like to be able to test aegir with vagrant
[07:03:58] <anarcat> vagrant should hit debian testing this week, so i'll install it at hme and may start playing with it
[07:04:09] <anarcat> other than that, don't count on me for vagrant stuff too much :)
[07:04:12] <anarcat> alright, that's it for me
[07:04:36] <ergonlogic> k, thanks anarcat
[07:04:43] <ergonlogic> I'll go next
[07:04:54] <anarcat> http://community.aegirproject.org/content/content/administrator/post-ins...
[07:04:58] <anarcat> go!
[07:05:10] <ergonlogic> I've updated aegir-up to run on the latest vagrant (0.9.8)
[07:05:28] <ergonlogic> and I've been using it to improve the puppet-aegir module
[07:06:00] <ergonlogic> so, I have plenty of experience with aegir running on vagrant, but none w/ Jenkins
[07:06:40] <ergonlogic> I've been poking around a bit
[07:07:17] <darthsteven> cool beans
[07:07:24] <ergonlogic> but I'd prefer to help someone with more jenkins experience, rather than try to do it myself
[07:07:50] <ergonlogic> anyway, we can now specify all the hostmaster variables in the puppet module
[07:08:00] <ergonlogic> and I'm currently working on supporting manual installs
[07:08:18] <ergonlogic> and I'm stuck at 'hostmaster-install' but I think I see a light at the end of the tunnel
[07:08:29] <ergonlogic> that's pretty much it for me
[07:09:11] <darthsteven> me next
[07:09:19] <darthsteven> I've had some time to fix some bugs!
[07:09:21] <darthsteven> yey!
[07:09:37] <anarcat> yay!
[07:09:41] <darthsteven> so 1.7 will have many fewer notices in the logs
[07:09:57] <darthsteven> and the site form now works in Internet Explorer
[07:09:58] <darthsteven> :)
[07:10:01] <anarcat> wow
[07:10:04] <ergonlogic> nice
[07:10:04] <anarcat> incredible
[07:10:07] <anarcat> IE6? :P
[07:10:12] <darthsteven> fixing some more simple bugs tonight
[07:10:17] <darthsteven> hah!
[07:10:19] <darthsteven> not likely
[07:10:21] <anarcat> sorry for the swearing ;)
[07:10:29] <anarcat> should we look at releasing 1.7 soon?
[07:10:30] <darthsteven> that's it from me
[07:11:17] <anarcat> it seems that omega8cc has a few patches sitting in the queue, i think mig5 looked at a few of those, but we should probably factor them in
[07:11:37] <anarcat> also, it seems to me that we should punish omega8cc with commit access, for too long they have been submitting good patches for nginx ;)
[07:11:37] <ergonlogic> if we do a 1.7 soon, I'd suggest we stick with Drupal 6.23, as 6.24 seems to have a couple bugs
[07:11:46] <anarcat> crap
[07:12:11] <omega8cc> heh, I'm still in the 'port hundreds of Nginx config changes from BOA fork to Aegir 1.x and 2.x' phase.
[07:12:14] <omega8cc> I didn't realize how far it got, but we will get it back in sync with our dear upstream again :)
[07:12:26] <anarcat> i am happy to hear that! :)
[07:12:30] <omega8cc> We are almost there, anyway.
[07:12:42] <anarcat> i am especially curious to see that "no create content stupid submenu" patch :)
[07:12:48] <omega8cc> 6.24 is fine with two patches
[07:12:52] <anarcat> that would be a great improvement
[07:13:03] <omega8cc> yeah
[07:13:05] <anarcat> we could ship with patches to the makefile...
[07:13:45] <anarcat> alright, anyone else?
[07:13:50] <ergonlogic> If we're going forward with a 1.7, I'll post an issue to track it
[07:14:16] <ergonlogic> omega8cc: if you could comment there with the patches, that would be great
[07:14:30] <anarcat> ergonlogic: let's
[07:14:37] <ergonlogic> I'm on it
[07:15:06] <anarcat> alright, anyone else has something to add to our scrum?
[07:15:16] <anarcat> otherwise i think we're done, thanks everyone for your attention
[07:15:57] <omega8cc> ergonlogic: basically: https://github.com/omega8cc/pressflow6/commit/97b8df2795e9565ffff5dcaa00... and https://github.com/omega8cc/pressflow6/commit/4b05a210c4234714104149c3ef...
[07:16:40] <omega8cc> there are two major issues only, both listed at http://drupal.org/drupal-7.12
[07:16:41] <anarcat> omega8cc: are those patches upstream?
[07:16:48] <ergonlogic> http://drupal.org/node/1439120
[07:16:48] <hefring> http://drupal.org/node/1439120 => Release 1.7 => Hostmaster (Aegir), Code, normal, active, 0 comments, 1 IRC mention
[07:17:35] <ergonlogic> I'll post the scrum log to c.a.o
Weekly Scrum IRC Log: 2012-02-20
[07:03:14] <mig5> scrum time anarcat darthsteven ergonlogic omega8cc
[07:03:58] <ergonlogic> hi
[07:03:58] <hefring> hi
[07:04:10] <mig5> i've not done much other than commit a patch here or there from others
[07:04:21] <mig5> we have a few RTBC that we should probably look at before the next release http://drupal.org/project/issues?text=&projects=hosting%2C+provision%2C+...
[07:04:36] <mig5> and a fair few for review http://drupal.org/project/issues?text=&projects=hosting%2C+provision%2C+...
[07:04:46] <darthsteven> I'm not able to make the scrum, sorry.
[07:05:15] <mig5> i suggest we also switch the aegir.make in the 6.x-1.x branch to start building drupal 6.23 or whatever, so we can be sure it works..
[07:05:25] <mig5> np thanks darthsteven
[07:05:57] <mig5> oh, we probably already build drupal 6.24 in the makefile. meh
[07:06:00] <ergonlogic> that makes sense
[07:06:11] <mig5> anyway, not much else from me
[07:06:33] <ergonlogic> for my part, I've been working on the puppet module
[07:06:39] <ergonlogic> and aegir-up
[07:06:58] <ergonlogic> so, we can now pass preseed info to the .deb
[07:07:15] <mig5> cool!
[07:07:20] <ergonlogic> and a number of othr parameters if we go with a 'manual' install
[07:08:03] <ergonlogic> and there's even a $aegir_dev_build variable that will build everything from the git repos, and preserver the .git repos
[07:08:35] <mig5> oh nice. i'm looking forward to using that :)
[07:08:39] <ergonlogic> :)
[07:09:08] <ergonlogic> I'll be going over most of that with anarcat this week, and whatever passes muster will go into the master branch
[07:09:28] <ergonlogic> that's most of what I've been up to
[07:09:42] <mig5> thanks ergonlogic
[07:09:57] <mig5> have we got anarcat here, or is he M.I.A this week
[07:10:12] <mig5> i'll keep it open for 5 or so min, in case omega8cc also arrives
[07:10:19] <omega8cc> hi
[07:10:26] <ergonlogic> I think he was pretty swamped today
[07:10:37] <omega8cc> I started new clean repos for both Provision and Hostmaster changes we have made in our BOA fork (it went too far out of sync in the meantime) and started submitting patches upstream, but it is just a start in a right direction.
[07:10:54] <mig5> great!
[07:10:56] <omega8cc> I plan to work more on that, this week. That is pretty much everything I have to report today.
[07:11:24] <mig5> omega8cc: see those RTBC tickets i linked above
[07:11:26] <mig5> a couple are nginx ones
[07:11:38] <mig5> should they be ignored in lieu of your new clean repos?
[07:11:42] <omega8cc> mig5: I will check all of them
[07:11:49] <mig5> thanks
[07:11:56] <mig5> they seem to be ssl specific
[07:11:59] <omega8cc> if they apply cleanly, no problem
[07:15:08] <ergonlogic> btw, undocumented aegir treasure of the week: passing '--working-copy' to hostmaster install will propagate down to 'drush make aegir.make'
[07:15:26] <ergonlogic> that is 'drush hostmaster-install'
[07:15:30] <anarcat> oh scrum
[07:15:35] <anarcat> missed that sry
[07:16:01] <ergonlogic> anarcat: go ahead
[07:18:11] <anarcat> hi
[07:18:11] <hefring> bonjour
[07:18:15] <anarcat> i want to release 1.7
[07:18:33] <mig5> \o/
[07:18:34] <anarcat> i will try to at least fixup the debian branches on wednesday, but i would also like to release 1.7
[07:19:06] * anarcat reading the backlog
[07:19:39] <anarcat> alright uh
[07:19:47] <anarcat> so yeah, i got ahead pretty good on that
[07:19:59] <anarcat> but the debian and release script cleanup will really take hold when we do that release
[07:20:06] <anarcat> so i'd like to do that, and improve the docs, in one swoop
[07:20:49] <anarcat> alright, i think that's it for me
[07:20:57] <anarcat> cheers everyone
[07:21:02] <ergonlogic> k, thanks
[07:21:08] <ergonlogic> anyone else?
[07:21:25] <ergonlogic> if not, I'll post the log to c.a.o
Weekly Scrum IRC Log: 2012-02-27
[07:00:23] <darthsteven> Scrum time?
[07:00:33] <mig5> yep
[07:01:30] <mig5> don't everyone go at once! :)
[07:01:37] <mig5> I can start
[07:01:45] <cmcintosh> by all means
[07:02:07] <anarcat> alright
[07:02:08] <mig5> not much from me, I committed a few patches mainly from omega8cc for nginx stuff, and a couple other small bugs. figured i'd get a few fixes in before 1.7
[07:02:17] <anarcat> scrum scrum scrum scrum scrumscrumscrum
[07:02:24] <darthsteven> Awesome stuff
[07:02:26] <mig5> go for it
[07:02:43] <cmcintosh> i have been working on building a integrated workflow using git / aegir tasks
[07:03:13] <anarcat> alright
[07:03:18] <cmcintosh> for now it will only have support for git but hoping to have api in it for addition of other types of repos
[07:03:36] <anarcat> that's cool
[07:03:42] <cmcintosh> yea
[07:03:48] <anarcat> alright, is that all?
[07:04:00] * anarcat is eager to go (haha)
[07:04:04] <cmcintosh> i also may be submitting a patch to tasks / platforms module to fix the issues i have been seeing with adding custom tasks into platform
[07:04:11] <cmcintosh> anarcat, go go go!
[07:04:14] <anarcat> alright
[07:04:16] <anarcat> i've been testing file descriptors overflow all weekend
[07:04:24] <darthsteven> Hehe
[07:04:27] <anarcat> running simulations creating billions of file descriptors
[07:04:32] <anarcat> and just now, it failed
[07:04:33] <anarcat> Resource id #2147480111
[07:04:34] <anarcat> 105 fd: Resource id #-2147483648, restartingstarting almost infinite loop to overflow PHP resources (file descriptors), press enter to continue or control-c to abort
[07:04:40] <anarcat> so drum roll please
[07:04:49] <anarcat> we'll now see if pcntl_exec() fixes the shit
[07:04:49] <darthsteven> Nice
[07:04:58] <anarcat> (drum roll?)
[07:04:59] <anarcat> whatever
[07:05:00] <anarcat> 105 fd: Resource id #-2147483648, restartingstarting almost infinite loop to overflow PHP resources (file descriptors), press enter to continue or control-c to abort
[07:05:03] <anarcat> Resource id #111
[07:05:06] <anarcat> Resource id #10111
[07:05:08] <anarcat> so there you go
[07:05:11] <anarcat> it fixes the shit!
[07:05:21] <anarcat> the fix is to re-exec the daemon, the bug doesn't survive pcntl_exec()
[07:05:38] <anarcat> so i think we could just re-exec the daemon after N tasks OR x seconds
[07:05:42] <darthsteven> Isn't that what we do already?
[07:05:49] <anarcat> well, we do it after a timeout
[07:06:01] <anarcat> my guess is that for a serie of heavy tasks that isn't enough
[07:06:06] <darthsteven> Ah right
[07:06:10] <mig5> so is this related to that http://drupal.org/node/1454316 ?
[07:06:11] <hefring> http://drupal.org/node/1454316 => Provision recurses infinitely on reading in context => Provision, Code, normal, active, 6 comments, 1 IRC mention
[07:06:12] <anarcat> say like 20 migrates
[07:06:15] <anarcat> mig5: no
[07:06:18] <mig5> ok
[07:06:20] <anarcat> it's in the hosting_queue_runner
[07:06:25] <mig5> oh right
[07:06:28] <anarcat> alright, so that was my crazy shit
[07:06:32] <anarcat> i want to release 1.7 soonish
[07:06:36] <anarcat> i am not sure i'll have time this week
[07:06:41] <anarcat> and besides we should wait for 6.25
[07:06:46] <mig5> yep
[07:06:50] <anarcat> ah
[07:06:53] <mig5> which is not far away thanks to that #drupalwtf
[07:07:01] <anarcat> and i want to introduce omega8cc as a new core committer
[07:07:18] <mig5> +1, we endlessly spoke of it but never did it :)
[07:07:25] <anarcat> we have already discussed this in the past, but i talked with omega8cc and we agreed that we could give commit access with the understanding that it is limited to nginx stuff
[07:07:26] <darthsteven> Cool
[07:07:31] <anarcat> which we can't really review anyways
[07:07:33] <mig5> i would prefer not to blindly commit patches with my current migression rate
[07:07:46] <anarcat> omega8cc: see? mig5 is just like you :P
[07:07:48] <anarcat> so
[07:07:54] <omega8cc> :)
[07:07:55] <anarcat> omega8cc: welcome in the team, and congratulations
[07:08:06] <anarcat> i think that's it for me, i can take care of christening the commit access
[07:08:07] <omega8cc> thank you! :)
[07:08:24] <anarcat> next!
[07:08:39] <darthsteven> omega8cc: Do you want to go next?
[07:09:07] <anarcat> omega8cc: your account is omega8cc on d.o?
[07:09:08] <omega8cc> darthsteven: thanks, I will be last
[07:09:12] <omega8cc> yes
[07:09:26] <anarcat> omega8cc: you now have access to provision
[07:09:32] <darthsteven> Okay, mig5 were you done?
[07:09:57] <anarcat> omega8cc: you now have access to hostmaster
[07:10:04] <mig5> darthsteven: yep
[07:10:19] <darthsteven> I'll go next then
[07:10:22] <darthsteven> :)
[07:10:35] <darthsteven> Right, I had a slightly busy weekend
[07:10:46] <anarcat> while i'm here, i removed tbosviel and univate from the maintainers ACL, as we haven't seen them forever
[07:11:07] <mig5> sure
[07:11:11] <darthsteven> And I've pushed forward the D7 rewrite
[07:11:29] <darthsteven> Note that I'm calling it a rewrite and not a port
[07:11:45] <mig5> awesome. I tried to drum up a bit of interest on twitter re: that
[07:11:48] <darthsteven> There's just too much baggage and D7 really does change the way that you build sites
[07:12:22] <darthsteven> There's quite a bit of discussion around entities vs. nodes already
[07:12:27] <anarcat> fair enough
[07:12:35] <anarcat> yeah, that's really interesting! i joined in there too
[07:12:49] <darthsteven> And so I want to facilitate that going forward, and reach consensus over the next few weeks
[07:12:52] <anarcat> i don't have strong opinions on the matter, as i am not familiar enough with d7, but i'm happy to see the talks move ahead
[07:12:59] <mig5> likewise ^^
[07:13:17] <cmcintosh> i think entities would be great
[07:13:23] <darthsteven> I'm targeting Drupalcon Europe for the release
[07:13:30] <anarcat> sweet
[07:13:31] <mig5> thanks for leading this darthsteven, it feels like it's got fresh air pumped into it again :)
[07:13:33] <anarcat> that would be awesome
[07:13:40] <cmcintosh> let me know i am getting some paid time for aegir dev and may be able to swing some time towards d7
[07:13:42] <darthsteven> So 6 or 18 months :p
[07:13:49] <anarcat> cmcintosh: that would be great
[07:13:52] <anarcat> haha
[07:14:03] <darthsteven> cmcintosh: That would be awesome
[07:14:10] <anarcat> darthsteven: oh and that would fit well with the offers for sponsorship for the subdir support
[07:14:13] <anarcat> did you guys see that?
[07:14:35] <cmcintosh> plus i need to sit down and upgrade aegir services for 3.x
[07:14:44] <darthsteven> anarcat: As you say in the issue, there needs to be community consensus on the rewrite, as otherwise no one will want to maintain it
[07:15:11] <darthsteven> cmcintosh: Hopefully, in D7 services integration will be so much easier
[07:15:24] <darthsteven> Thats it from me I think
[07:15:33] <cmcintosh> sounds awesome
[07:15:35] <darthsteven> Would happily chat nodes vs entities at some point
[07:15:45] <darthsteven> (at a pub at the moment)
[07:15:45] <anarcat> darthsteven: well, my point is more that the person that will do the upgrade should have the final say
[07:15:52] <anarcat> as i expect it will be you :P
[07:16:00] <cmcintosh> darthsteven, you got any code snipets for adding tasks to platforms on d6, would be great to review to see if im doing things right here
[07:16:37] <mig5> cmcintosh: did you give an aegir talk at Sandcamp last month?
[07:16:40] <darthsteven> cmcintosh: I think it was the Drupal gardens importer, which is on d.o somewhere
[07:16:53] <anarcat> so i wanted to add this http://drupal.org/node/705026#comment-5655840
[07:16:54] <hefring> http://drupal.org/node/705026 => Allow creation of mysite.com/site1 and mysite.com/site2 type of sites => Hostmaster (Aegir), Code, major, needs work, 29 comments, 1 IRC mention
[07:17:05] <anarcat> it looks like we have people interested in sponsoring proper subdir support implementations
[07:17:10] <cmcintosh> no i was going to but didnt make it
[07:17:13] <anarcat> and that would fit well with darthsteven's timeline for d7
[07:17:41] <darthsteven> anarcat: If you could get the backend able to do that, I could get a frontend ship shape to drive it
[07:17:48] <anarcat> that would be awesome
[07:18:05] <anarcat> i have zero cycles now, esp. since i am looking at 1.7 and sites dir improvements
[07:18:09] <mig5> how would we do it.. with 'Alias' in vhosts or something?
[07:18:12] <anarcat> but it's certainly something we need
[07:18:13] <darthsteven> Though I have some ideas of how it could work in the backend too :)
[07:18:14] <anarcat> mig5: yeah.
[07:18:23] <anarcat> i think that's the best idea so far
[07:18:25] <mig5> i can't fathom how we'll be able to do migrations of sites, e.g if they are sharing the one vhost ServerName
[07:18:33] <mig5> but need to be on separate platforms
[07:18:40] <anarcat> aliases do that
[07:18:40] <mig5> i guess we'd have to just maintain the individual Alias somehow
[07:18:43] <mig5> yeah
[07:18:44] <anarcat> yeah
[07:18:56] <mig5> anyway - i think this is falling outside scrum, shall we tie it off?
[07:18:59] <mig5> anything to add ergonlogic ?
[07:19:01] <mig5> oh!
[07:19:02] <mig5> omega8cc! :)
[07:19:08] <mig5> don't think you've had your turn
[07:19:08] <anarcat> yeah!
[07:19:17] <omega8cc> I will take this opportunity and will finally sync Nginx stuff between BOA fork and our upstream in the next 2-3 days, but it will require a small flood of Nginx related patches. So we should have nicely polished Nginx config before 1.7 release.
[07:19:31] <ergonlogic> oops, sorry 'bout that
[07:19:39] <omega8cc> Thanks for the commit access!
[07:19:47] <mig5> we forgot to make sure you wanted it! :)
[07:19:48] <omega8cc> I will do my best
[07:20:07] <anarcat> mig5: i did ask before the scrum :)
[07:20:13] <anarcat> mig5: which was the key bit we were missing :)
[07:20:17] <mig5> oh good
[07:20:19] <omega8cc> yes :)
[07:20:23] <mig5> 'they forced me into core dev' sniff
[07:20:28] <anarcat> haha
[07:20:30] <omega8cc> haha
[07:21:20] <ergonlogic> for my part, I'm planning on a beta release of aegir-up this week, and to start migrating hostmaster.profile to the Profiler library
[07:21:24] <ergonlogic> that's it
[07:21:35] <ergonlogic> sorry for missing it :-/
[07:21:41] <mig5> great stuff. will the hostmaster profile rewrite be for 6.x-2.x or just the 7.x port?
[07:21:46] <cmcintosh> darthsteven, can you get me a link to the dl for that module
[07:21:53] <cmcintosh> doesnt seem to be anything in the repo on d.o
[07:22:42] <darthsteven> cmcintosh: http://drupal.org/sandbox/darthsteven/1178192
[07:22:57] <ergonlogic> both 6.x and 7.x, I figure
[07:23:14] <darthsteven> cmcintosh: http://drupalcode.org/sandbox/darthsteven/1178192.git/tree/refs/heads/6....
[07:23:26] <mig5> alright I think we'll call that a scrum
[07:23:32] <darthsteven> Yup
[07:23:33] <mig5> I'll do a fresh Aegir Newsletter today I think
[07:23:36] <darthsteven> Thanks guys
[07:23:38] <mig5> thanks all
Weekly Scrum IRC Log: 2012-03-05
[07:00:08] <@darthsteven> anarcat, darthsteven, mig5, EclipseGc, ergonlogic, mvc, omega8cc, shrop, skwashd: Scrum time!
[07:01:48] <@darthsteven> anyone else here?
[07:03:00] <@omega8cc> ah, it's Monday again?
[07:03:56] <@darthsteven> it is!
[07:04:07] <@darthsteven> wouldd you like to go firs this week?
[07:04:13] <@darthsteven> *first
[07:06:09] <@iler> :)
[07:06:33] <@darthsteven> no?
[07:06:38] <@darthsteven> okay well I'll go then
[07:06:48] <@anarcat> scrum
[07:06:52] <@anarcat> sry
[07:06:56] <@darthsteven> Hello!
[07:06:56] <@hefring> what's up
[07:07:09] * penyaskito has quit (Ping timeout: 252 seconds)
[07:07:10] <@darthsteven> So last Friday I spent the day working on the D7 port
[07:07:35] <@darthsteven> lots of initial looking at entity module and getting some of our entities set up
[07:07:53] <@darthsteven> nothing major to report, only that it's going to take a while to port :)
[07:08:23] <@darthsteven> I added a couple of command line options to our mysqldump command
[07:08:36] <@darthsteven> and broke aegir doing so, but then fixed it, so all is well
[07:08:40] <@darthsteven> that's it from me?
[07:08:47] <@darthsteven> anarcat: do you want to go next?
[07:09:01] <@darthsteven> (that is it from me, btw)
[07:09:25] * scor has joined #aegir
[07:09:35] <@anarcat> yep
[07:09:38] <@anarcat> so
[07:09:57] <@anarcat> i'll work on the 1.7 release tomorrow
[07:10:03] <@anarcat> i hope to have time to wrap it up
[07:10:14] <@anarcat> i think omega8cc merged their patches in so the timing seems ripe
[07:10:17] <@darthsteven> cool
[07:10:21] <@scor> http://community.aegirproject.org/content/installing/automatic-installat... says the current Ubuntu Server LTS is 11.04. however according to https://wiki.ubuntu.com/LTS it does not seem that 11.04 is LTS
[07:10:25] * svendecabooter has quit (Remote host closed the connection)
[07:10:38] <@iler> scor: It's not, 10.04 is
[07:10:54] <@scor> so this documentation applies to 10.04?
[07:10:55] <@iler> And next one is 12.04
[07:10:56] <@anarcat> koumbit reprioritised its roadmap and we'll look at migrate optimisations next, trying to consolidate the files/ directory outside of the migrate
[07:11:07] <@darthsteven> scor/iler: we're mid-scrum please keep chatter to a minimum until after (in about 10 mins)
[07:11:10] <@anarcat> then we'll look at subsite support
[07:11:19] <@scor> darthsteven: oops sorry will come back later
[07:11:27] <@darthsteven> anarcat: awesome
[07:11:55] <@anarcat> then we'll look at securing verify from code injection. e.g. by not running cc all and by looking directly in the system table instead of bootstrapping the site
[07:12:11] <@anarcat> so that remote acces to the sites dir is more secure
[07:12:18] <@anarcat> so that's our roadmap for the next year
[07:12:21] <@omega8cc> nice
[07:12:39] <@anarcat> most of those will be done on 2.x, and we'll start deploying those for willing guinea pigs
[07:12:39] <@darthsteven> lovely
[07:12:50] <@anarcat> so i'll probably setup a 2.x debian archive too
[07:13:21] <@anarcat> i feel sick last week so i didn't do much work other than that planning
[07:13:25] <@anarcat> that's it for me
[07:13:35] <@darthsteven> awesome, thanks anarcat
[07:13:37] <@darthsteven> omega8cc?
[07:13:40] <@omega8cc> ok
[07:13:51] <@omega8cc> Last week I merged in all Nginx related fixes and improvements we already have tested and used in BOA, and also fixed some missing Nginx config bits between 6.x-1.x and 6.x-2.x.
[07:14:04] <@omega8cc> This week I plan to commit some UX improvements in hostmaster we have in BOA for a long time already, in a separate branch of course, for review.
[07:14:19] <@anarcat> cool
[07:14:20] <@omega8cc> and that is it from me, I think.
[07:14:22] * q-rban is now known as q0rban
[07:14:34] <@darthsteven> awesome work omega8cc
[07:14:45] <@darthsteven> I've been getting lots of emails about issues being updated/fixed
[07:14:47] <@anarcat> omega8cc: you think the stuff you merged is ready for release? :)
[07:15:11] <@omega8cc> anarcat: haha, it is used in production for months!
[07:15:23] <@anarcat> cool :)
[07:15:33] <@omega8cc> darthsteven: thanks, I promised the flood :)
[07:15:33] <@anarcat> omega8cc: oh, and i saw you're looking at working on nginx debian packaging
[07:15:39] * patcon has joined #aegir
[07:15:40] <@omega8cc> yes
[07:15:47] <@anarcat> omega8cc: we may want to sync up on that as i will work on having easier to build 2.x packages
[07:15:53] <@anarcat> so that would be the right place to dev that
[07:16:04] <@omega8cc> I agree
[07:16:11] <@anarcat> expect that tomorrow
[07:16:22] <@anarcat> alright anything else?
[07:16:25] <@anarcat> we're hitting our timebox
[07:16:42] <@anarcat> i know that ergonlogic has proposed to split hostmaster in eldir/hosting/hostmaster parts back again
[07:16:49] <@anarcat> i'd be curious to hear what ppl think of that
[07:17:09] <@darthsteven> it makes sense for Eldir, but not for hosting
[07:17:36] <@anarcat> well, there's an issue opened about his
[07:17:41] <@darthsteven> indeed :)
[07:17:57] <@darthsteven> end of scrum unless anyone else wants to chip in in the next 15 seconds...
[07:18:38] <@darthsteven> that's a scrum then folks
Weekly Scrum IRC Log: 2012-03-12
[07:01:35] <darthsteven> scrum time!
[07:03:44] <ergonlogic> hi
[07:04:30] <darthsteven> hello!
[07:04:30] <hefring> salut
[07:04:37] <omega8cc> hi
[07:04:37] <hefring> hello
[07:04:56] <omega8cc> hefring: botsnack
[07:04:56] <hefring> thanks omega8cc :)
[07:05:39] * scientist_ has joined #aegir
[07:05:50] <ergonlogic> I believe anarcat is on vacation, and has found his way out to the woods
[07:05:55] <darthsteven> mig5: scrum?
[07:06:02] <ergonlogic> so he won't likely be joining us
[07:06:20] <darthsteven> I shall kick things off by saying well done to Anarcat for getting the 1.7 release done
[07:06:31] <darthsteven> and well done to omega8cc for getting tons of stuff into that release
[07:06:42] <darthsteven> that's it from me
[07:06:52] <omega8cc> :)
[07:07:11] <omega8cc> My work on porting Hostmaster BOA UI improvements to 6.x-2.x is still in progress, so I'm submitting some minor Nginx patches in the meantime. Not much to report this week.
[07:08:21] <omega8cc> However, I'm really tired with this bug: http://drupal.org/node/1004526
[07:08:22] <hefring> http://drupal.org/node/1004526 => Automatic aliases are not persisted across rename and clone => Hostmaster (Aegir), Code, major, needs work, 25 comments, 3 IRC mentions
[07:08:33] <darthsteven> ah yes, that one
[07:08:34] <omega8cc> it causes tons of support requests
[07:08:44] <omega8cc> so I plan to attack it
[07:09:20] <darthsteven> ok cool
[07:09:36] <omega8cc> it is a major Aegir WTF/fail for most of our users
[07:11:49] * SqyD has joined #aegir
[07:12:25] * penyaskito has joined #aegir
[07:12:45] <darthsteven> indeed
[07:12:49] <darthsteven> anything else?
[07:12:58] <omega8cc> that is it from me
[07:13:09] <ergonlogic> For my part I submitted some issues and patches to port Hostmaster to the Profiler library, making sub-profiles possible, and fairly easy. This also required a ptch to Provision to support alternative profile names.
[07:14:00] <ergonlogic> I also depends on splitting Eldir and Hosting back out of Hostmaster for the 6.x-2.x branch, and I submitted an issue on that as well
[07:14:23] <darthsteven> yeah, I'm not 100% sure where I feel about a split tbh
[07:14:34] <ergonlogic> ok
[07:15:03] <darthsteven> it's a good idea for us, but not sure if it will confuse people, and lead to lots of where should I post this issue type questions?
[07:15:15] <darthsteven> but I can see the benefits too
[07:15:21] <ergonlogic> well, we already get lots of that
[07:15:57] <ergonlogic> I'm pretty sure it will confise some people
[07:16:08] <ergonlogic> but the other projects already exist
[07:16:56] <darthsteven> yup
[07:17:04] <darthsteven> anyway…anyone got anything else to add?
[07:17:18] <ergonlogic> Beyond making hosting and eldir reusable, I think it could also make engagement in the project easier
[07:17:18] * Artusamak_afk is now known as Artusamak
[07:18:55] <darthsteven> possibly
[07:18:57] <ergonlogic> anyway, I's also been making lots of headway with Aegir-up, which now supports full development builds of Aegir, including custom (i.e. sandbox) provision repos and alternative branches
[07:19:05] <darthsteven> awesome
[07:19:19] <darthsteven> got a link for that work?
[07:19:23] <ergonlogic> along with custom aegir.make makefiles
[07:20:07] <ergonlogic> a bit short on docs, but http://community.openatria.com/team/node/552
[07:20:29] <ergonlogic> Aegir-up itself is at http://drupal.org/project/aegir-up
[07:20:50] <ergonlogic> that's about it from me
[07:21:13] <apassi> I have a aegir installed on two server, now when i create server content type from other to other, then aegir ui does not work anymore on other server, why?
[07:21:43] <darthsteven> thanks ergonlogic
[07:21:52] <darthsteven> i think we'll call that a scrum then
[07:22:02] <darthsteven> anyone care to do the logs?
[07:22:06] <ergonlogic> I can
[07:22:20] <darthsteven> ergonlogic: thanks
[07:22:34] <ergonlogic> darthsteven: can we discuss splitting hosting/eldir?
[07:22:47] <darthsteven> ergonlogic: sure, can you give me ten mins?
[07:22:47] <ergonlogic> darthsteven: I'd like to make sure I understand your reservations
[07:22:52] <ergonlogic> sure
[07:23:10] <ergonlogic> hefring: logs
[07:23:21] <ergonlogic> gah, what is that command again?
[07:23:27] <ergonlogic> hefring: timestamp
[07:23:31] <darthsteven> hefring: log pointer?
[07:23:31] <hefring> http://hefring.mig5.net/bot/log/aegir/2012-03-12#T182454
[07:23:42] <ergonlogic> right... thanks
Weekly Scrum IRC Log: 2012-03-19
[06:47:48] <darthsteven> hefring: Tell anarcat that I'll be unable to make the scrum tonight but I've only done one issue, which was getting MySQL passwords escaped properly when installing, so in theory we can use special characters in root passwords just fine.
[06:47:49] <hefring> darthsteven: I'll pass that on when anarcat is around.
[06:51:19] * penyaskito has quit (Quit: Saliendo)
[07:01:31] <mig5> scrum time
[07:01:36] <mig5> thanks darthsteven for the message
[07:02:05] <mig5> nothing from me this week! suspect it will be a quiet scrum
[07:02:15] <mig5> anarcat: omega8cc: ergonlogic: go for it
[07:02:50] <ergonlogic> yep
[07:03:31] <ergonlogic> for my part, I wrote a cron queue for CiviCRM
[07:04:02] <ergonlogic> http://drupal.org/project/hosting_civicrm_cron
[07:04:11] <anarcat> sorry, i gotta go
[07:04:11] <hefring> anarcat: 16 min 23 sec ago <darthsteven> Tell anarcat that I'll be unable to make the scrum tonight but I've only done one issue, which was getting MySQL passwords escaped properly when installing, so in theory we can use special characters in root passwords just fine.
[07:04:30] <ergonlogic> as part of last week's Koumbit provision_civicrm sprint
[07:04:51] <anarcat> nothing much to say from me anyways, 1.7 was released i am happy, and i will work on optimise migrate next
[07:04:53] <ergonlogic> and I continue to work on aegir-up
[07:04:54] <anarcat> ciao!
[07:05:03] <anarcat> !po
[07:05:04] <ergonlogic> that's it for me
[07:05:15] <mig5> thanks guys
[07:05:40] <mig5> i'll keep it open for 5 min in case omega8cc drops in
[07:06:26] <omega8cc> hi!
[07:06:26] <hefring> yo
[07:06:30] <omega8cc> While no progress on my end in regard to UI improvements ported from BOA Hostmaster yet, I have found an easy way to manage extra ftps/lshell accounts per Aegir client with access strictly limited to attached sites, everything done in a one bash script outside of Aegir, and it will be included in BOA-2.0.3
[07:07:17] <mig5> sounds interesting
[07:07:20] <omega8cc> and that is the end of news this week :)
[07:08:08] <omega8cc> the script: http://drupalcode.org/project/barracuda.git/blob/HEAD:/aegir/tools/auto-...
[07:10:20] <mig5> alright, thanks all!
[07:10:31] <omega8cc> it creates/purges extra accounts using built-in lshell features, it doesn't sync with Aegir passwords, so it gives extra control over granting and revoking access (manually)
[07:10:38] <omega8cc> thanks!
Weekly Scrum IRC Log: 2012-03-26
[06:59:46] <ergonlogic> mig5 omega8cc: scrum?
[07:00:14] <ergonlogic> darthsteven & anarcat indicated they wouldn't be able to make it today
[07:00:57] <omega8cc> ergonlogic: we may need to re-schedule it then?
[07:01:20] <ergonlogic> yep, anarcat had asked to re-schedule for an hour earlier
[07:02:05] <omega8cc> ergonlogic: makes sense also here, as it is now 10pm cet, not sure about mig5?
[07:02:44] <mig5> hi there
[07:02:44] <hefring> mig5: 2 hours 39 min ago <darthsteven> Tell mig5 that I'll be unable to make the scrum again, I've worked on fixing some bugs, including one major one that I introduced in 1.7. I would suggest that we should release 1.8 soonish. I also did some work on aegir-up to make it super useful for developing aegir itself.
[07:03:09] <mig5> i am OK for it to be rescheduled - however, if it moves earlier in the day, i won't be showing up :)
[07:03:14] <mig5> and i think that's ok, to be honest...
[07:03:25] <mig5> we've spent years to trying to accomodate for everyone and i think it's the wrong approach
[07:03:44] <mig5> i'll send updates via hefring in advance, and read scrum notes afterward
[07:03:45] <ergonlogic> mig5: ok
[07:03:54] <ergonlogic> mig5: do you have a suggestion?
[07:04:09] <ergonlogic> nobody wants to exclude anyone else
[07:04:14] <mig5> nope. any later is too late for the europeans
[07:04:24] <mig5> any earlier, i will not wake up (hard enough already :) )
[07:04:42] <mig5> and by the time the europeans are getting up, it's late for me, so same problem
[07:04:56] <mig5> reversed
[07:04:59] <ergonlogic> yep
[07:05:06] <ergonlogic> no wins
[07:05:22] <omega8cc> mig5: hah, hefring could help a bit, but it is then not really interactive :) 22pm cet is still ok for me
[07:05:56] <mig5> i'm not fussed on 'exclusion', it's simply the nature of things :) i'd sooner see four or 5 of you talking, via logs, than three of us
[07:06:35] <mig5> nothing new from me anyway this week - darthsteven has been amazing in the issue queues, fixed a lot of things. and it looks like i need to finally check out aegir-up
[07:06:59] <omega8cc> or maybe let's schedule two scrums on Monday/Tuesday then? so we could choose which scrum to attend?
[07:07:02] <ergonlogic> well, we do alot of communicating outside the scrums, so I don;t think it's essential
[07:07:22] <ergonlogic> omega8cc: that's another good idea
[07:07:35] <mig5> yes a morning/night scrum could be interesting :)
[07:07:47] <ergonlogic> though maybe monday/thursday, or something
[07:07:54] <ergonlogic> so it's spread out a bit more?
[07:08:02] <omega8cc> sometimes I work late in the night cet
[07:08:03] <mig5> yep
[07:08:05] <omega8cc> good idea
[07:08:20] <ergonlogic> yeah, same here (late nights)
[07:09:11] <omega8cc> so let's convert the old problem into this new idea
[07:09:35] <ergonlogic> so, monday afternoons, one hour earlier
[07:09:48] <ergonlogic> and thursdays when it's convenient for mig5?
[07:10:31] <ergonlogic> later, we could perhaps schedule the monday ones earlier to be easier on the europeans...
[07:10:44] <omega8cc> sure, is thursday ok for you mig5?
[07:10:48] <mig5> i'm confused on when is 'monday afternoon' - afternoon for who?
[07:10:57] <ergonlogic> re aegir-up, I'min the process of turning it into a drush extension
[07:11:00] <mig5> thursday is fine for me - at this time, or the other end of the day?
[07:11:16] <ergonlogic> mig5: well, for me, obviously :P
[07:12:06] <omega8cc> mig5: be the host of the thursday scrum and choose the hour :)
[07:12:21] <ergonlogic> mig5: I meant one hour earlier than current time on mondays (for now)
[07:12:30] <ergonlogic> for the Monday scrums
[07:12:56] <mig5> yes, go one hour earlier on the mondays - as i'll likely just drop out of those, so make it convenient for you
[07:13:12] <ergonlogic> right
[07:14:27] <mig5> and perhaps we go the opposite for Thursday
[07:14:27] <mig5> http://www.timeanddate.com/worldclock/meetingdetails.html?year=2012&mont...
[07:14:47] <mig5> so it's the time in montreal for oyu that is for me right now, but europeans and asia-pacific can join in
[07:15:36] <mig5> or perhaps one hour earlier, same as you're going to make it for monday
[07:15:49] <mig5> http://www.timeanddate.com/worldclock/meetingdetails.html?year=2012&mont...
[07:16:18] <omega8cc> this will work for me, but maybe 9pm would be better
[07:16:21] <omega8cc> right
[07:16:55] <mig5> ok sure
[07:17:17] <ergonlogic> alright, I've update the monday scrums on c.a.o
[07:17:38] <ergonlogic> I'll leave it to one of you to add the second scrum time?
[07:17:57] <omega8cc> mig5: :)
[07:18:18] <ergonlogic> I guess we can just remove 'weekly' from those pages now :)
[07:18:29] <omega8cc> yep
[07:20:00] <ergonlogic> hefring tell darthsteven we're moving the Monday scrums up by 1 hour (for now) as per anarcat's request, and starting a second one on Thursdays, to accomodate Mig5 and Omega8cc
[07:20:00] <hefring> ergonlogic: I'll pass that on when darthsteven is around.
[07:20:20] <ergonlogic> hefring tell anarcat we're moving the Monday scrums up by 1 hour (for now) as per your request, and starting a second one on Thursdays, to accomodate Mig5 and Omega8cc
[07:20:20] <hefring> ergonlogic: I'll pass that on when anarcat is around.
[07:21:02] <omega8cc> I still need to take a look at this aegir-up stuff, but am I correct you are rewriting it as a drush ext?
[07:21:10] <ergonlogic> k, so, let's wrap this up, unless anyone had anything substantive?
[07:21:14] <ergonlogic> omega8cc: yep
[07:21:27] <omega8cc> awesome!
[07:21:27] <ergonlogic> the basics are in place
[07:21:39] <ergonlogic> with drush wrappers around the shell scripts I'd written
[07:21:57] <ergonlogic> but I'll start new commands natively in drush, and port the rest as time permits
[07:22:31] <ergonlogic> hefring: log pointer
[07:22:31] <hefring> http://hefring.mig5.net/bot/log/aegir/2012-03-26#T187514
[07:22:39] <omega8cc> I have very little to report this week, basically a bit of simple magic in Aegir and Drush 4.6-dev to make it work with Drupal 8
[07:22:53] <ergonlogic> omega8cc: yeah, that was nice!
[07:23:17] <ergonlogic> I haven't tried it out yet, but nice to see!
[07:23:42] <omega8cc> that was not really a magic, rather boring search/replace, but still, it is nice it works
[07:24:01] <ergonlogic> drush 5 was released
[07:24:33] <ergonlogic> so we might want to start moving aegir to it
[07:24:43] <omega8cc> yeah, but it drops drupal 5 support and we are not compatible yet, afaik
[07:24:58] <ergonlogic> there are a couple issues about it, yeah
[07:25:04] <omega8cc> so we have a choice in the meantime
[07:25:36] <ergonlogic> well, I guess that can be one of the differentiators for 2.x? no more D5?
[07:27:07] <omega8cc> yeah, however we have still people on pressflow 5, but it will be a good selling point to give them EOL for d5 support in Aegir
[07:27:31] <ergonlogic> drush also supports COMMANDFILE.drush4.inc, or something similar
[07:28:08] <ergonlogic> so we could keep our current functionality for backward compatibility
[07:28:16] <omega8cc> didn't know that, nice
[07:28:19] <ergonlogic> but it would mean alot more code to maintain
[07:28:38] <omega8cc> yeah, not worth the effort probably
[07:29:04] <ergonlogic> it's an option, if we want to deprecate D5 support, I guess
[07:29:59] <ergonlogic> either way drush 5 is the future, and allows for some nifty things
[07:30:02] <ergonlogic> anyway
[07:30:12] <ergonlogic> I'll post the log
Weekly Scrum IRC Log: 2012-04-09
[05:02:08] <ergonlogic> anarcat mig5 omega8cc darthsteven everyone: scrum?
[05:02:34] <omega8cc> hi
[05:02:34] <hefring> what's up
[05:02:45] <anarcat> hi
[05:02:45] <hefring> eh oh
[05:02:48] <omega8cc> same hour? i think we have moved it, no?
[05:02:56] <omega8cc> oh
[05:02:57] <ergonlogic> we'd moved it to this time
[05:03:04] <ergonlogic> from an hour later
[05:03:28] <omega8cc> ah, so it is still 9pm cet, k
[05:04:19] <ergonlogic> so, there appears to have been lots of activity in the issue queues these past couple weeks...
[05:04:33] <ergonlogic> anyone want to speak to that?
[05:04:49] <ergonlogic> or other stuff we've been working on?
[05:05:24] <omega8cc> well, I worked on a workarounds/fixes for two issues
[05:05:38] <omega8cc> both a year old
[05:05:48] <omega8cc> the famous http://drupal.org/node/1004526
[05:05:49] <hefring> http://drupal.org/node/1004526 => Automatic aliases are not persisted across rename and clone => Provision, Code, major, needs review, 32 comments, 4 IRC mentions
[05:06:00] <omega8cc> and another: http://drupal.org/node/1088472
[05:06:01] <hefring> http://drupal.org/node/1088472 => Introduce simple switch for changing write permissions to support plugin manager (Drupal App Store) => Provision, Code, normal, needs review, 17 comments, 3 IRC mentions
[05:06:48] <omega8cc> patches for review submitted
[05:07:05] <ergonlogic> great!
[05:07:19] <omega8cc> however, they are not really fixes, just a workarounds
[05:08:18] <anarcat> alright that's good
[05:08:28] <omega8cc> now I'm packing BOA as a deb monster so also Nginx based Aegir installed from deb should be available soon
[05:08:31] <anarcat> i don't think i'll have time to review this now, but it's good to see some progress on that
[05:08:40] <omega8cc> ok
[05:08:52] <anarcat> omega8cc: someone was asking about nginx + debian here earlier, you may want to join forces
[05:08:55] <omega8cc> and that is it from me
[05:09:02] <anarcat> cweagans it was
[05:09:37] <omega8cc> anarcat: it should be easy, but any help is appreciated, sure!
[05:09:42] <omega8cc> right
[05:09:56] <anarcat> as for myself, not much to say here
[05:10:20] <anarcat> i am looking forward to having more time to work on aegir, but we are doing major infrastructure work on our old cluster here and it's taking a lot of manhours
[05:10:35] <anarcat> then i will look at optimising migrate
[05:10:47] <anarcat> and doing an estimate for those folks who want to chip in to implement subsites
[05:12:45] <millette> salut
[05:12:45] <hefring> hello
[05:12:47] <ergonlogic> anarcat: k, anything else?
[05:12:55] <anarcat> uh i think that's it for me
[05:12:59] <ergonlogic> hi millette!
[05:13:13] <millette> anarcat, could you provide me with an aegir install somewhere so I can do my packages patch properly?
[05:13:21] <ergonlogic> millette: we're just in the middle of our weekly scrum
[05:13:32] <millette> salut ergonlogic - ack - I'll stay tuned
[05:13:59] <ergonlogic> for my part, I've finished porting Aegir-up to a drush extension now
[05:14:29] <ergonlogic> and I'vebegun work on a utility server to provide dns, squid-proxy, git and jenkins servers
[05:14:41] <ergonlogic> so I'll be poking around the jenkins test scripts
[05:14:51] <ergonlogic> that's about it from me
[05:15:10] <ergonlogic> omega8cc: was there a Thursday scrum?
[05:15:49] <omega8cc> ergonlogic: no, I was here but I didn't met anyone here :/
[05:16:07] <ergonlogic> omega8cc: alright
[05:16:27] <ergonlogic> so then, if there's nothing else, I'll post this to the community site
[05:16:36] <flexgrip> Has anyone enabled CORS on an nginx/BOA install before?
[05:16:39] <ergonlogic> hefring: log pointer
[05:16:39] <hefring> http://hefring.mig5.net/bot/log/aegir/2012-04-09#T192145
[05:16:40] <anarcat> alirght
[05:16:42] <anarcat> thanks ergonlogic
[05:16:46] <omega8cc> thanks
[05:16:54] <ergonlogic> :)
Weekly Scrum IRC Log: 2012-04-12
[21:01:31] <mig5> alright! i actually made it, almost on time, to the second scrum of the week
[21:01:39] <mig5> i missed the first few weeks, and i'm meant to be the one hosting it. epic
[21:01:50] <mig5> darthsteven: anarcat: ergonlogic: if you're around
[21:02:04] <darthsteven> mig5: hello!
[21:02:37] <mig5> i updated the python-cloudfiles installation on the jenkins server, and raised a sleep timeout limit that was causing all our jenkins builds to fail, as rackspace takes ages to build from our images these days sadly
[21:02:55] <darthsteven> cool
[21:03:07] <darthsteven> not cool about rackspace though
[21:03:11] <mig5> i then suckered myself into working more on what was meant to be a trivial patch, and now has left me bewildered and confused http://drupal.org/node/1503824
[21:03:12] <hefring> http://drupal.org/node/1503824 => hosting_site_count() ignores disabled sites => Hostmaster (Aegir), Code, normal, needs work, 5 comments, 1 IRC mention
[21:03:16] <mig5> like the last 20 or so patches over the last 2 years
[21:03:24] <mig5> that's about it from me!
[21:03:46] * rv0 has quit (Read error: Connection reset by peer)
[21:03:54] <mig5> some of our builds in jenkins are still broken, this seems to be related to installing deb files via dpkg, which i haven't figured out yet
[21:04:26] <mig5> weird requirements like nfs-common, and the drush-make deb, etc.. and my attempt to fix it saw that restarting portmap or something failed. haven't had time to look at.
[21:06:05] <darthsteven> sorry if I've added to your confusion in that issue
[21:06:25] <mig5> nah it's not you. any developer would be fine with it
[21:06:43] <mig5> iam just not made of the same stuff and never have been. i should find something i'm good at and stick with it :)
[21:06:54] <mig5> don't have a coder brain
[21:07:50] <darthsteven> okay, well the efforts are appreciated
[21:07:54] <darthsteven> shall I go next?
[21:08:06] <mig5> please!
[21:08:17] <darthsteven> well, I've looked at a couple of issues in the queue
[21:08:52] <darthsteven> notably: 1185690
[21:09:10] <darthsteven> which is running through jenkins now
[21:09:13] <darthsteven> a minor fix
[21:09:34] <darthsteven> We'll release 1.8 next week
[21:09:38] <darthsteven> I think
[21:09:42] <darthsteven> that's it from me
[21:09:59] <mig5> nice, thanks
[21:11:07] <mig5> real shame that we lost data from the community site
[21:11:14] <mig5> niccolo wrote something interesting in response to http://community.aegirproject.org/discuss/wheres-aegir-drupalcon-denver
[21:11:36] <mig5> about how it feels like the project is sort of losing its spark, so to speak, in the wake of other beasts like pantheon
[21:11:45] <mig5> i've been wondering about that and how we can get that spark back
[21:12:02] <darthsteven> indeed
[21:12:13] <mig5> it's probably in the eye of the beholder, of course.. some might feel it's very new and exciting and others less so
[21:12:33] <mig5> but i don't think we ever fully recovered from the developmentseed departure
[21:12:40] <darthsteven> yup
[21:13:41] <mig5> I haven't got much to suggest, given I myself struggle with immense motivation issues. ho, hum
[21:13:55] <darthsteven> I'm now working at least a day week, at work, on Aegir
[21:14:00] <mig5> that's unreal :)
[21:14:26] <mig5> i wanted to ask how you currently do your deployments etc
[21:14:31] <darthsteven> so hoping to get some of the longer standing bugs sorted out
[21:15:19] <darthsteven> and some of the more complicated bugs that would take a few hours to set up multiple servers etc.
Weekly Scrum IRC Log: 2012-04-16
[04:59:05] <ergonlogic> darthsteven anarcat mig5 omega8cc: scrum?
[04:59:33] <anarcat> yep, i can
[04:59:43] <ergonlogic> k, great
[05:00:03] <ergonlogic> was a Thursday scrum last week, which is good to see
[05:01:06] <ergonlogic> discussion of 1.8 release this week?
[05:01:14] <anarcat> cool
[05:01:22] <anarcat> the thursday scrum is at 7hAM is that right?
[05:01:31] <anarcat> i am worried about the dual scrums
[05:01:45] <ergonlogic> anarcat: I think it's like 4am here in Mtl
[05:01:52] <anarcat> i feel it's splitting the community
[05:01:57] <anarcat> it's 6am here
[05:02:09] <ergonlogic> ah, ok, well, no difference to me :)
[05:02:25] <anarcat> so anyways
[05:02:30] <anarcat> i really think we should release 1.8 soon
[05:02:35] <anarcat> i am not sure i will have time to do this now
[05:02:42] <anarcat> but maybe we can call for patches before the release
[05:02:47] <ergonlogic> since we log them, I think we'll be ok, as long as we bother to read the ones we miss
[05:02:52] <anarcat> i have worked on provisionacl last week
[05:03:02] <anarcat> and i have made it so it changes the mode on the whole sites directory
[05:03:10] <anarcat> so that users can create local.settings.php or .git and so on
[05:03:17] <anarcat> this fixes a bunch of issues at koumbit
[05:03:21] <ergonlogic> sweet
[05:03:28] <anarcat> but makes securing aegir harder
[05:03:46] <anarcat> i still have to do that estime for the subsites thing
[05:04:38] <anarcat> and i am looking at moving files/modules/... out of the sites directory, maybe even making the whole directory a symlink
[05:04:51] <anarcat> that's about it for me
[05:05:03] <ergonlogic> moving them to clients/?
[05:05:21] <anarcat> yeah, basically
[05:05:38] <ergonlogic> that'd probably simplify sftp and such
[05:06:05] <ergonlogic> ok, well on my part, I've been granted commit access to drush-vagrant
[05:06:23] <ergonlogic> so I'm migrating a bunch of code from aegir-up over there
[05:06:43] <ergonlogic> which should help to stabilize aegir-up somewhat
[05:07:05] <ergonlogic> as I tease apart the templating framework for vagrant, and the template that deploys Aegir
[05:07:55] <anarcat> that's good
[05:07:56] <ergonlogic> anyway, that and getting our AegirVPS infrastructure worted out thave been keeping me busy
[05:08:20] <ergonlogic> that latter, I might blog about in a week or two, once I've got it running the way I'd like
[05:08:29] <anarcat> cool
[05:08:44] <ergonlogic> and so that aegir-up can create a working local environment that more or less matches it
[05:09:18] <ergonlogic> I've started a second template to work similarly to aegir-up, that includes a Jenkins server
[05:10:22] <ergonlogic> it's also planned to have bind, squid and git servers
[05:11:08] <ergonlogic> so it can be the local end of a continuous integration workflow, but it depends on drush-vagrant stabilizing a bit more first
[05:11:17] <ergonlogic> anyway, that's pretty much it from me
[05:11:48] <ergonlogic> darthsteven omega8cc mig5 others: anything to add?
[05:12:15] <omega8cc> I fixed a few small bugs in the nginx config so it should now just work with dotdeb build, as it should be recommended build for nginx based installs on Debian, then started to review the code missing in 2.x, as there are patches committed only in 1.x, however so far only 3 are tagged, but I will review also previous months to find them all
[05:12:34] <omega8cc> http://drupal.org/project/issues?text=&projects=provision%2C+hosting%2C+...
[05:13:27] <omega8cc> then I will move to the deb based build, finally
[05:13:47] <omega8cc> I mean, to make the deb install working with nginx
[05:14:13] <omega8cc> and that is it from me, I guess
[05:14:56] <ergonlogic> darthsteven, mig5, anyone else?
[05:15:33] <ergonlogic> Office hours are worth noting
[05:15:40] <omega8cc> right
[05:15:46] <ergonlogic> http://community.aegirproject.org/content/office-hours
[05:15:49] <omega8cc> great idea
[05:15:59] <ergonlogic> agreed
[05:16:51] <ergonlogic> anarcat: aha, that's what's scheduled for 4:00am :)
[05:17:06] <ergonlogic> anyway, I'll post the log
[05:17:10] <ergonlogic> hefring: log pointer
Weekly Scrum IRC Log: 2012-04-26
20:06 <+omega8cc> scrum? 20:06 < hefring> Every Monday at 20h00 UTC: 12h00 San Francisco, USA (PDT), 15h00 Montreal, Canada (EDT), 20h00 London, UK (GMT), 07h00 (Tuesday) Melbourne, Australia (EST) 20:08 <@mig5> oops 20:08 <@mig5> in an hour i think? 20:08 <+omega8cc> no, now! :) 20:09 <+omega8cc> mig5: is this 21h00 in Melbourne, Australia (EST)? 20:09 <@mig5> hmm, it's 20h00 here 20:09 <+omega8cc> heh 20:09 <+omega8cc> so we need to fix this http://community.aegirproject.org/scrums 20:09 <@mig5> maybe i moved back and the rest of you stayed where oyu are 20:09 <+omega8cc> as it says 12h00 (noon) in Warsaw, Poland 20:09 <@mig5> now is fine, if that's the normal time 20:09 <@mig5> i think i must've moved back 20:09 <@mig5> so it was EDT before 20:10 <@mig5> ok, scrum :P 20:10 <+omega8cc> ah 20:10 <@mig5> i have nothing to add anyway! 20:11 <+omega8cc> Nothing new to report from me today, but I do have an idea I want to discuss later. 20:11 <+omega8cc> We have some concept-grade, half-baked modules for importing the site into Aegir (or migrating between Aegir instances), plus many how-tos etc. But what we really need is an aegir_connector, like in this demo: http://vimeo.com/41020017 This is what I want to get done right in the next month or two. 20:13 <+omega8cc> mig5: so we need to fix the hours at http://community.aegirproject.org/scrums as it should much your time when it is a good time in AU 20:13 <+omega8cc> match* 20:13 <@mig5> now is fine, i'll just change melbourne's time 20:14 <+omega8cc> ok 20:15 <@mig5> i can only add that i played with the Pack module and managed to get it working. i think the docs for Pack need a lot of work, and there are some 'gotchas' such as the aegir UID/GID needing to match between systems or aegir fails to chmod/chown during rsyncs etc 20:15 <@mig5> and, i wrote some neat Jenkins/Fabric magic that automatically tags install profile repositories and updates stub makefiles and does tag-based deployment with drush make etc 20:15 <@mig5> for a client 20:15 <@mig5> hope to release it soon 20:16 <+omega8cc> wow 20:16 <@mig5> finally i hope to do another aegir newsletter soon, it's been a while 20:16 <+omega8cc> sounds great 20:16 <@mig5> when i say 'updates stub makefiles' i mean, updated the [tag] = xxxxx for that install profile referenced in the stub makefile 20:17 <@mig5> in otherwords allows a user to manually kick off a 'Build' in jenkins and it does the whole tagged release.. standard provision-migrate etc but with proper tags etc. 20:18 <@mig5> anyways, small scrum i think. i think we also didn't have one on tuesday, too bad.. 20:18 <+omega8cc> mig5: you remember Vertice idea to have makefiles subscribed via some feed and auto-generating new platforms? 20:18 <@mig5> yeah 20:18 <@mig5> that would be neat 20:19 <@mig5> you'd need something faster than cron to subscribe to it though 20:19 <@mig5> jenkins is faster atm 20:19 <@mig5> maybe, erm, node.js :) /me runs 20:20 <+omega8cc> haha, I think we need more stuff like that, to help adoption, as all those Hpowerful workflows with Jenkins etc are just over the head of 99% of the audience we could attract 20:20 <@mig5> of course 20:22 <+omega8cc> and that is my plan (and it always has been) to make it easy for people who don't really have a chance to be a power users, but want the benefits of using the Aegir stuff with their mouse only (almost) 20:24 <+omega8cc> because, you know, power users prefer command line and ci stuff etc anyway, so they will never really help in Aegir adoption, because power users are always a small percent of the audience 20:24 <@mig5> yes it's been good that you stick to your audience :) 20:24 <@mig5> the real world 20:24 <+omega8cc> both are real 20:25 <+omega8cc> just a different scale 20:27 <+omega8cc> ok 20:27 <+omega8cc> hefring: botsnack 20:27 < hefring> delicious! 20:27 <+omega8cc> scrum
Weekly Scrum IRC Log: 2012-04-30
[05:01:07] <anarcat> darthsteven / mig5 / EclipseGc / ergonlogic / mvc / omega8cc / sfyn / shrop / skwashd : scrum time!
[05:02:33] <anarcat> ... anyone around? :)
[05:02:33] <darthsteven> awesome
[05:02:36] <anarcat> yay!
[05:02:36] <ergonlogic> hi
[05:02:40] <anarcat> hi
[05:02:40] <hefring> eh oh
[05:02:52] <omega8cc> hi
[05:02:52] <hefring> bonjour
[05:02:53] <anarcat> hefring freakout
[05:02:59] <anarcat> hey omega8cc
[05:03:04] <anarcat> alright, who wants to start?
[05:03:12] * feizing has joined #aegir
[05:03:28] <darthsteven> Who wants to go first?
[05:04:09] <anarcat> i can go, i don't have much to say, i'm afraid
[05:04:25] <anarcat> i made a new release of provisionacl that is much more liberal about file permissions
[05:04:32] <shrop> hello
[05:04:54] <anarcat> we recovered from a catastrophic operator failure on our aegir install 2 weeks ago (drush @hostmaster < wrongdb.sql => lots of fun)
[05:05:01] <ergonlogic> hi shrop
[05:05:17] <shrop> ergonlogic: hey there :)
[05:05:21] <anarcat> which outlined a few bugs in import and that critical SSL bug that was fixed in 1.8 (thanks darthsteven !)
[05:05:44] <anarcat> i'm still looking at finding time to do the estimate for subsites support
[05:05:53] <anarcat> that's about it for me i think
[05:07:00] <darthsteven> thanks
[05:07:26] <darthsteven> my irc connection is not so great, so I'll go next
[05:08:25] <darthsteven> I got 6.x-2.x working with Drush 5
[05:08:39] <darthsteven> which was reasonably easy
[05:08:54] <darthsteven> but, there are a couple of upstream bugs that need fixing in Drush first
[05:09:09] <darthsteven> http://drupal.org/node/1554092 is particularly nasty for us
[05:09:10] <hefring> http://drupal.org/node/1554092 => drush_invoke_process uses 100% CPU => Drush, Base system (internal API), critical, needs review, 2 comments, 2 IRC mentions
[05:09:24] <anarcat> wtf
[05:09:44] <darthsteven> not sure how Drush 5 was released with that issue, but there we go
[05:10:05] <darthsteven> hopefully that'll get fixed soon, I've posted a patch
[05:10:16] * feizingDROID has joined #aegir
[05:10:16] <darthsteven> I pushed a few other fixes in 6.x-1.x
[05:10:41] <darthsteven> including my fix for seg-faults/infinite loops when using contexts
[05:10:58] <darthsteven> hopefully ergonlogic will tell me that it fixed his issue earlier
[05:11:04] <darthsteven> that's it from me!
[05:11:12] <anarcat> niiice
[05:11:28] <ergonlogic> darthsteven: well, it gave a more useful error message, but in the end it was a case of PEBKAC
[05:11:29] <anarcat> you could mention you singlehandedly released yet another awesome aegir release? :)
[05:11:46] <anarcat> ergonlogic: most of those are PEBKAC, the idea is exactly to give a useful error message :)
[05:11:56] <darthsteven> anarcat: wasn't that the week before :)
[05:12:01] <ergonlogic> then the patch worked worderfully!
[05:12:03] * anarcat out of it ;)
[05:12:05] <darthsteven> ergonlogic: can we chat afterward?
[05:12:14] <ergonlogic> darthsteven: of course
[05:12:16] <darthsteven> next!
[05:12:19] <darthsteven> ergonlogic?
[05:12:25] <ergonlogic> ok
[05:12:37] * David_Hernandez has quit (Quit: Saliendo)
[05:12:47] <ergonlogic> well, I updated api.aegirproject.org today with the new branches and such
[05:12:55] <ergonlogic> thanks for the poke darthsteven
[05:13:05] <ergonlogic> I'd been meaning to do that for awhile
[05:13:40] <ergonlogic> otherwise falling a little too deep doe the rabbit hole of Drush development
[05:14:16] <anarcat> oh boy :)
[05:14:19] <ergonlogic> I'd love to see provision's context stuff split off into it's own extension
[05:14:31] <ergonlogic> as I'm rolling my own aliases fro drush-vagrant
[05:14:55] <ergonlogic> and I tried to grok how to add a new context type
[05:15:05] <ergonlogic> but well, I couldn't
[05:15:25] <ergonlogic> and a dependency on provision seemed a little steep
[05:15:42] <ergonlogic> anyway, that's most of it for me
[05:16:01] <ergonlogic> oh, I also noticed that there isn;t a drush 5 git branch
[05:16:17] <ergonlogic> everything is going into master
[05:17:02] <anarcat> i am not sure that drush 6 devel has started
[05:17:08] <darthsteven> ergonlogic: did you moshe's response: http://drupal.org/node/1554762
[05:17:09] <hefring> http://drupal.org/node/1554762 => Clarify git branches => Drush, Miscellaneous, normal, fixed, 2 comments, 1 IRC mention
[05:17:31] <ergonlogic> just saw it now
[05:17:42] <anarcat> wtf
[05:17:53] <anarcat> how can the releases be any different if they are on the same branch?
[05:18:48] <ergonlogic> everything goes into drush 5 until they have to break the API?
[05:18:54] <ergonlogic> hmm, ok
[05:19:45] <anarcat> alright
[05:19:47] <anarcat> you done ergonlogic ?
[05:19:50] <ergonlogic> yep
[05:19:52] <anarcat> omega8cc: you next?
[05:20:05] <omega8cc> I have fixed a few issues in the Nginx config and made it compatible with 1.1.8 and later, so also 1.2.0, on the fly
[05:20:12] <omega8cc> Also, I have discovered this new critical, which caused enough wtf issues (broken configs) on hostmaster upgrade for Nginx users in the past: http://drupal.org/node/1552430
[05:20:13] <hefring> http://drupal.org/node/1552430 => Hostmaster upgrade/migrate doesn't update @server_master drush alias and web server config files => Provision, Code, critical, needs review, 8 comments, 1 IRC mention
[05:20:22] <omega8cc> And that's it from me, today.
[05:20:49] <anarcat> mind if i prod you for the debian package? :)
[05:21:01] <anarcat> where is the development happening?
[05:21:15] <anarcat> also, i have given you autoop here, since well, you are core now :)
[05:21:19] <omega8cc> hah, still in the works
[05:21:24] <omega8cc> ah
[05:21:30] <omega8cc> thanks :)
[05:21:59] <omega8cc> and "still in the works" means no progress, in this case :/
[05:22:04] * simg has left #aegir ()
[05:22:06] <anarcat> alright
[05:22:23] <anarcat> well, if you would push your work somewhere, people could jump in and give a hand...
[05:22:35] <anarcat> i'd certainly be curious to see the work
[05:22:36] <omega8cc> sure, will doo
[05:23:01] <omega8cc> this week, promise :)
[05:23:16] <omega8cc> we also need this, finally
[05:23:33] <omega8cc> so chances are it will happen
[05:24:04] <anarcat> awesome
[05:24:12] <anarcat> darthsteven: i commented on that loop bug
[05:24:26] <anarcat> i think that is all for our scrum
[05:24:37] <anarcat> unless there are other additions, i think we'll wrap it up here
[05:24:50] * feizingDROID has quit (Quit: Bye)
[05:25:27] <darthsteven> that is all
Weekly Scrum IRC Log: 2012-05-10
20:01 <@mig5> scrum time! 20:02 <@mig5> darthsteven: anarcat: ergonlogic 20:02 <@darthsteven> Hello! 20:02 < hefring> salut 20:03 <@mig5> i've recently started using the pack feature, as a very large client of mine will probably use it soon, so i decided to eat the dogfood and transitioned my own infrastructure onto a 5-server high-availability environment :) 20:03 <@mig5> my main issue has been http://drupal.org/node/1555398 20:03 < hefring> http://drupal.org/node/1555398 => MySQL GRANTs aren't given for multiple servers in Web Pack => Hostmaster (Aegir), Code, normal, active, 18 comments, 1 IRC mention 20:03 <@mig5> we have some funny corner cases with mysql grants 20:03 <@mig5> not actually limited to pack module, but to a case where a webserver might have multiple ip addresses in the same LAN assigned to it 20:03 <@mig5> and mysql might pick the wrong *source* IP to make the mysql connection attempt in order to fetch those to use for grants 20:04 <@mig5> i've worked around it in my case there. but anyway that's basically all i'm thinking about at the moment 20:07 <@mig5> darthsteven: can we catch up after the scrum? 20:07 <@mig5> presuming the scrum isn't done already :) 20:07 <@darthsteven> (we can) 20:08 <@darthsteven> that's great 20:08 <@darthsteven> I saw lots of comments on that issue :) 20:08 <@darthsteven> shall I go next? 20:08 <@mig5> yeah i got a bit carried away - but it was really hard to find *what* was the problem 20:08 <@mig5> sure 20:09 <@darthsteven> didn't really do too much this week 20:09 <@darthsteven> got a fix for the Drush invoke issue in Drush 5 in: http://drupal.org/node/1554092 20:09 < hefring> http://drupal.org/node/1554092 => drush_invoke_process uses 100% CPU => Drush, Base system (internal API), critical, fixed, 13 comments, 3 IRC mentions 20:10 <@darthsteven> and working with the Drush peeps to get http://drupal.org/node/1550574 sorted too 20:10 < hefring> http://drupal.org/node/1550574 => drush_invoke_process doesn't pass through alias => Drush, Base system (internal API), normal, active, 4 comments, 1 IRC mention 20:11 <@darthsteven> we should be able to support Drush 5 easily when that one is done 20:11 <@darthsteven> at the moment the 6.x-2.x branch has a workaround 20:11 <@darthsteven> but 6.x-2.x still works with Drush 5 20:11 <@darthsteven> that's it from me! 20:11 <@darthsteven> next? 20:11 <@mig5> nice! 20:12 <@mig5> i suspect that'll be it, it's only 6:12am in montreal 20:12 <@mig5> i'll wait a few min 20:20 <@mig5> ok i think that's it 20:20 <@mig5> thanks 20:22 <@darthsteven> thank you too
Weekly Scrum IRC Log: 2012-05-28
[05:02:37] <@darthsteven> anarcat, mig5, omega8cc, EclipseGc, mvc, sfyn, shrop, skwashd: scrum? [05:02:43] <@anarcat> yay [05:03:49] <@darthsteven> shall I go first? [05:03:55] <@darthsteven> I've done not a lot [05:03:58] <@shrop> Not at my computer, but will watch from mobile. Cheers! [05:04:09] <@darthsteven> released 1.9 a little while ago [05:04:23] <@anarcat> upgrade went through like a breeze here [05:04:24] <@darthsteven> that's it from me! [05:04:30] <@anarcat> alright [05:04:31] <@darthsteven> next! [05:04:43] <@anarcat> well, for me, thanks to omega8cc's grant, i have been able to free up more time for the project [05:04:53] <@anarcat> my first focus was on urgent drush security issues [05:05:07] <@anarcat> http://drupal.org/node/671906 [05:05:09] <@hefring> http://drupal.org/node/671906 => mysql credentials leak in drush sqlc => Drush, PM (dl, en, up ...), critical, needs review, 44 comments, 3 IRC mentions [05:05:33] <@anarcat> this is mostly fixed, but i have found deep issues in the sql code and other credentials leaks that are not yet fixd [05:05:44] <@anarcat> my progress can be followed here: http://drupal.org/user/1274/track [05:06:01] <@anarcat> and here: http://drupal.org/user/1274/track/code [05:06:32] <@anarcat> which reminds me: i have started work on a pretty dumb merge script for apache log files [05:06:35] <@anarcat> http://drupal.org/project/provision-mergelog [05:06:51] <@anarcat> it is designed to suck apache logs from remote servers into a single file that can then be parsed by logfile analysis software [05:07:11] <@anarcat> i have done some tests with awstats, and it works well, considering awstats haven't been updated in 3 years [05:07:17] <@anarcat> btw, piwik sucks for that [05:07:21] <@darthsteven> cool! [05:07:31] <@anarcat> while it *can* process those logfiles, it's not reliable at all and makes *ONE HIT PER LINE* on the webserver [05:07:42] <@anarcat> which is a lot of fun if the piwik site is in the logfile itself :P [05:08:03] <@anarcat> i have also dug my head in the SSL code again, and found quite a lot of issues [05:08:14] <@anarcat> found new bugs: [05:08:15] <@anarcat> http://drupal.org/node/1603722 [05:08:15] <@hefring> http://drupal.org/node/1603722 => deleting a site doesn't delete its SSL certificate => Hostmaster (Aegir), Code, normal, needs work, 2 comments, 1 IRC mention [05:08:32] <@anarcat> and updated the existing one: http://drupal.org/node/1126640 [05:08:33] <@hefring> http://drupal.org/node/1126640 => move the SSL IP allocation to the frontend => Hostmaster (Aegir), Code, normal, needs work, 20 comments, 1 IRC mention [05:08:40] <@anarcat> but we need to redo the SSL code, it's quite crappy right now [05:08:52] <@anarcat> i am not sure we want to ditch IP management, as SNI doesn't seem quite mature enough yet [05:08:59] <@anarcat> so i'd like to support both SNI and non-SNI setups [05:09:19] <@anarcat> alright what else have i done [05:09:40] <@anarcat> well, i guess all is left is what next: [05:09:51] <@anarcat> i will work on improving migrate performance [05:10:09] <@anarcat> https://drupal.org/node/1484214 [05:10:19] <@anarcat> i will attack https://drupal.org/node/1205458 shortly [05:10:24] <@anarcat> and i'm looking for advice on how to do that [05:10:32] <@anarcat> should files reside in the clients directory? [05:11:00] <@anarcat> anyways, that's about it for me [05:11:14] <@anarcat> i am going for a trip in sao paulo brasil between june 5th and 18th [05:11:26] <@anarcat> if people want to hookup or organize a little aegir meetup there, that could be fun and useful [05:11:32] <@anarcat> next up? [05:12:35] <@anarcat> that was quite a monologue [05:12:52] <@darthsteven> (all good stuff though) [05:13:52] <@darthsteven> anyone else? [05:14:20] <@omega8cc> not much from me to report, other that we have now nice ssd machine for project testing etc [05:15:12] <@darthsteven> indeed, thanks very much for that! [05:15:30] <@anarcat> cool [05:15:35] <@anarcat> who's following up on that setup? [05:15:59] <@darthsteven> I am [05:16:07] <@anarcat> alright [05:16:09] <@anarcat> that's awesome [05:16:18] <@anarcat> i noticed that ergonlogic posted something about vagrant [05:16:21] <@darthsteven> should be able to get it sorted this week or next [05:16:25] <@anarcat> he showed me his work, it's pretty good [05:16:26] <@omega8cc> ah, I have added r1soft backups there, just in case [05:16:40] <@darthsteven> I'm trying to 'do it right' and set it all up using puppet [05:18:10] <@anarcat> that's awesome [05:19:32] <@darthsteven> we'll call that a scrum there [05:19:34] <@darthsteven> oh [05:20:11] <@darthsteven> actually, Drush 5 got itself a bit more sorted, and so Aegir could probably be released for Drush 5 now, if we wanted, but we'll have a better release if we wait a bit [05:20:25] <@darthsteven> but we'll call it a scrum now.
3.0 roadmap
We do not commit to doing all of this for 3.0, this is mostly a wishlist of what people expressed were priorities for them during the brainstorm. If people want things to move forward, they will need to submit patches or create feature branches, basically lead development on the feature you want. This page can be used to coordinate overall leadership of those issues, while pointing to the Drupal.org issue queues for the specifics.
This roadmap supersedes the previous 2.0 roadmap.
We'll start bumping items from the 2.0 roadmap over here...
- Drupal 7 ( or 8) port of the frontend
Port frontend to Drupal 7
We were originally planning to release 1.0 with Drupal 7 support to make sure it was a "long term support" release, but this has changed - we hope to keep the release cycle short and therefore Drupal 7 was pushed back to 2.0. But it is still a target to make sure we a reliable release that will be supported for a long time.
Drupal 7 also introduces the concept of entities which could be very useful to lighten our many content types in the frontend. It could also be useful for DNS support as zonefiles could be modeled directly as entities.
This concerns of course only the port of the frontend (hostmaster.profile, eldir.theme and hosting.module), the backend already supports d7 installs and upgrades. The issue to followup is #1261030.
Aegir core team
The Aegir maintainers are committers that have access to the code. They are responsible for specific parts of the project and are general go-to people for various things.
- cweagans - maintainer of a number of contrib modules, and contributor to several more, as well as patches to Aegir core.
- ergonlogic - Aegir Project Leader, release engineering, continuous integration testing, community site webmaster, documentation, general bugfixing, ubercart integration co-maintainer, SaaS contrib components
- helmo - hairy bugfixes and generally great contributions in the queue, maintainer for several contrib components.
- Jon Pugh - maintainer of devshop hosting/provision, provision_git, and tons of other contrib modules, submitter of many good patches
- omega8cc - nginx support Provision Sandbox, Barracuda Project
Emeritus members
These members have decided to take a less active role in the project. However, out of respect for them, and all their previous efforts for the Aegir Project (and the hope that they'll become more active again ;), they continue to have commit access.
- anarcat - debian packaging, crazy late night coding splurges, general goto guy for everything
- mig5 - testing, release engineering, migressions and general bugfixing
- Steven Jones - documentation, queueing and contexts stuff, Drupal 7 port, general bugfixing
Other contributors
Those people have given significant contributions to the project but do not yet have complete commit access.
- sfyn - ubercart integration, quota API
- Deciphered - views integration and other patches
- gboudrias - Contributor of a number of contrib modules, kplatforms and patches to Aegir core.
Sorry if your name isn't here and you have done significant contributions!! Feel free to add it or contact us if you feel you have been forgotten...
All contributors of Aegir's history can also be found on Ohloh.net's contributors page although that doesn't count patches that were sent in the Drupal.org issue queues.
Past contributors
- vertice - project founder, retired
- tbosviel - DNS support improvements
- univate - views support and batch operations
- seth.vincent - Google Summer of Code student, working on D7 port and DNS editor. Sandbox
How to join the core team
If you are interested in joining the core team, you should provide us with valuable patches or contribute to the issue queues significantly. The core team has access to all core git repositories for writing, so you will need to prove that you can provide good quality code through the issue queue. We usually notice such people and welcome them in the team directly, but you should also feel free to ask to join if you feel you are worthy.
Aegir Co-maintainer Criteria
The Aegir core team is somewhat selective about inviting co-maintainers. This is largely due to the complexity of the project, and the crucial functionality it plays for many of its users.
Below we detail the responsibilities of co-maintainership (apart from just commit rights to git), as well as what criteria we feel are important for new team members. Co-maintainership involves taking responsibility for the project's overall well-being. This includes:
- Participation in the community: issue queues, IRC, community site, etc.
- Technical involvement: bug-fixing, patch testing, new feature development and roadmap/architecture planning, release engineering, etc.
- Maintenance of project infrastructure: debian repos, community and API sites, Jenkins, etc.
- Community outreach: presentations and BoFs at Camps and Cons, blogging, screencasts, etc.
While this isn't an exhaustive list, nor are all these activities required for membership in the core team. We're looking for new team members with regular, productive activity in one or more of the above domains. This essentially boils down to two criteria:
- code/patches submissions (quality over quantity) and
- demonstrated interest in the project.
If you believe that you'd make a good addition to the core team, don't hesitate to contact any of us to discuss it.
Aegir Project Leadership
With the recent transfer of leadership, the project's core team has decided to move towards a more democratic model, to ensure clearer structure and smoother transitions in the future. Debian is a shining example of this (in particular, the DPL), and so we will model our leadership position(s) after them.
The Aegir Project Leader (APL) is the official representative of the Aegir Project. They have two main functions, one internal and one external.
In the external function, the Project Leader represents the Aegir Project to others. This involves giving talks and presentations about Aegir and attending trade shows, as well as building good relationships with other organizations and companies.
Internally, the Project Leader manages the project and defines its vision. They should talk to other Aegir developers, especially to active contributors to Aegir core, to see how they can assist their work. A main task of the Project Leader therefore involves coordination and communication.
Obviously, we're a much smaller project than Debian, and so we're adopting a pretty minimalist organizational structure. That said, Debian has many other structures in place to ensure the ongoing health of a large community driven project. As the Aegir Project grows, we can appropriate additional tools to overcome any new challenges that arise.
Finally, the responsibilities inherent in the APL role are not exclusive to it. That is, just because it is the Project Leader's job to give presentations and coordinate our efforts, other community members should not hesitate to continue doing so themselves.
How to welcome someone to the core team
Once someone is deemed worthy of joining the core team, you (a core member) should make a proposal on the core mailing list to welcome the new developer. Subject should clearly state the proposal and give ample time for other developers to chime in. Usually all core team members should give their explicit approval as we try to reach consensus (and usually do) over proposals.
Once the member is approved on the mailing list, you should follow this checklist:
- give membership on the mailing list
- grant commit access to the core repositories (Hostmaster, Hosting, Provision and Eldir)
- grant jenkins access (simply create a user then go to the configuration page to give him the same access as others - consider also Vagrant scripts used in Jenkins and maybe the Aegir Jenkins scripts
- grant operator status on the IRC channel (
/msg ChanServ flags #aegir user +*
) -user
needs to be registered by the member first - grant admin access to the community site
- add the person to the core team page
- send a nice welcome email to the new member and CC the mailing list
- optionnally: announce the great news in the next release
Development news
There's an aggregated feed of all commits to the stable branches of Aegir available through yahoo pipes:
http://pipes.yahoo.com/pipes/pipe.info?_id=05014cd510f5bececc20212b20843117
Office hours
Starting from April 2012, we will be holding 'office hours' twice weekly, these are opportunities when people knowledgeable about Aegir will be around to support the community.
1. When
Office hours currently take place on:
- Every Monday at 14h00 - 16h00 UTC (Convert this to my timezone)
- Every Thursday at 08h00 - 10h00 UTC (Convert this to my timezone)
2. Where
These occur via IRC on the #aegir channel, you can read about how to get on IRC on the Drupal.org IRC page. You don't need to install an IRC client on your machine to join, you can chat through your web browser.
3. What
We can discuss anything Aegir related, but ideally we'd use this time primarily for the following:
- Helping others to contribute to the Aegir project.
- Fixing bugs in the Aegir project.
- Adding and clarifying documentation.
If you want to just pop along and be assigned something to do, just announce in the #aegir channel that you'd like to partake in office hours, and we'll take it from there!
3.0 release notes
The Aegir team is very pleased to announce the official release of Aegir 3.0. This release ships with significant stability improvements, Drush 6 and 7 support, improved SSL, subdirectory multisite and nginx support and much, much more!
This release marks the deprecation of the 2.x branch. We will continue to provide security releases through the end of 2015. All Aegir users are strongly encouraged to upgrade to the 3.0 release. The upgrade has been thoroughly tested and has worked reliably for us.
Upgrading from Aegir 2.x moves Aegir from a Drupal 6 platform to a Drupal 7 one. The Debian package will migrate the front-end for you. While we have tested this with Aegir core, we strongly recommend disabling any contributed modules you may be running.
We have a very dynamic community of contrib developers, and various projects built atop Aegir. So if you were waiting for Aegir 3 to be stable, easy to install and production-ready, now is the time. The 3.x branch will be well supported as a lot of shops are running it in production already.
1. Major changes
The front-end has been ported to Drupal 7. We've also raised the minimum Drush version to 6.x, and the Debian package now installs Drush via Composer. Drush 7 is also supported.
We've added several "Golden contrib" modules to the distribution:
- Hosting Remote Import
- Hosting Site Backup Manager
- Hosting Git
- Hosting Tasks Extra
We're planning to add additional well-supported contrib modules over the course of the Aegir 3.x lifecycle. Along similar lines, we've removed hosting_platform_pathauto, as it's built-in now.
We've introduced initial Drupal 8 support, though this is still a work-in-progress. This mostly depends on Drupal 8 and Drush 8 stabilizing, before we'll be able to more fully manage Drupal 8 sites.
Aegir 3 now supports SNI for SSL.
A number of significant bug fixes and stability improvements have been incorporated. Among them, we've added locking in our task queue, and improved backup reliability significantly.
For more information, please see the slidedeck from the presentation by ergonlogic and helmo at the Aegir Summit: http://community.aegirproject.org/sites/community.aegirproject.org/files...
2. Major API changes
The API has not significantly changed, though there are a few additions. See the current API docs in the various repositories for additional details.
3. Installing and upgrading
The canonical source of installation documentation is
http://docs.aegirproject.org/en/latest/install/install/
We're in the process of converting our docs, the old version is on http://community.aegirproject.org/installing We're tracking the conversion on https://www.drupal.org/node/2534734
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
Note that you should upgrade to the latest 2.x release (currently: 2.4) before attempting the upgrade to 3.x..
It is still imperative that you read the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
4. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
5. Features and improvements
In addition to the "major changes" mentioned above, the following should also be noted.
Changes since 7.x-3.0-rc1:
- #2494121: Stop servers in clusters from sync'ing back to master.
- Make the upgrade script slightly more verbose
- #2377819: Gzipping backups suppresses file permissions errors
6. Known issues
- Drupal 8 support is (Chasing HEAD) not complete. You would need Drush 8 for that.
- Drush master has an issue on Debian Wheezy (https://www.drupal.org/node/2491941)
- When upgrading from 6.x-2.x enable the betterlogin module manually.
3.0-rc1 release notes
The Aegir team is pleased to announce the first release candidate of the Aegir 3.x branch, after months of development. Development has been sometimes intermittent, but it is not over yet!
This release is tailored to make upcoming changes available to a wider audience than people ready to install from git, but also to make features more widely available and tested.
In the issue queues the central roadmap issue is: #1261030 [meta] Roadmap: Aegir 3.x (D7 port)
1. Major changes
- None, mostly small bugfixes
2. Major API changes
to be determined.
3. Installing and upgrading
The canonical source of installation documentation is
http://docs.aegirproject.org/en/latest/install.html
We're in the process of converting our docs, the old version is on http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
Note that you should upgrade to the latest 2.x release (currently: 2.4) before attempting the upgrade to 3.x..
It is still imperative that you read the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
4. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
5. Features and improvements
In addition to the "major changes" mentioned above, the following should also be noted.
Changes to hostmaster since 7.x-3.0-beta2:
Changes to provision since 7.x-3.0-beta2:
- #2530904 by PascalAnimateur: Unknown package version in Drupal 7
- #2530904 by PascalAnimateur, ybabel: Package version always set to Unknown
- #2511772 by danwonac - Nginx: Restore support for legacy rewrites non-clean URLs
- #2497091 - The replacement string in the third line needs to be literal
- Clarify regex alter function API docs.
- Replace drupal_set_message by drush_log
- #2496417: Drupal 6 and PHP 5.6 (Debian Jessie)
- Match Drush 8
- Set port properly when generating temporary .my.cnf.
Changes to hosting since 7.x-3.0-beta2:
- #2507089 by gboudrias: Dependency on overlay?
- #2446749 by Jon Pugh: Improve hosting task drush hooks and status update
6. Known issues
- Drupal 8 support is (Chasing HEAD) not complete.
- Drush master has an issue on Debian Wheezy (https://www.drupal.org/node/2491941)
3.0-beta2 release notes
The Aegir team is pleased to announce the forth preview release (beta) of the Aegir 3.x branch, after months of development. Development has been sometimes intermittent, but it is not over yet!
This release is tailored to make upcoming changes available to a wider audience than people ready to install from git, but also to make features more widely available and tested. More beta releases will be published when there are substantial changes.
In the issue queues the central roadmap issue is: #1261030 [meta] Roadmap: Aegir 3.x (D7 port)
1. Major changes
- Fixed a security issue in the protection of the files directory for multi-site.
- Improved SSL support
- Both Drush stable (6.6.0) and 7.x (the master branch) are supported.
2. Major API changes
to be determined.
3. Installing and upgrading
The canonical source of installation documentation is
http://docs.aegirproject.org/en/latest/install.html
We're in the process of converting our docs, the old version is on http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
Note that you should upgrade to the latest 2.x release (currently: 2.4) before attempting the upgrade to 3.x..
It is still imperative that you read the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
4. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
5. Features and improvements
In addition to the "major changes" mentioned above, the following should also be noted.
Changes ho hostmaster since 7.x-3.0-beta1:
- Add hosting_remote_import to the distro.
- Update the views version in our makefile.
- Remove hosting_platform_pathauto, as it's built-in now.
Changes to provision since 7.x-3.0-beta1:
- Fix protection of files dir for multi-site
- #2036333: Remove CVS deploy code.
- Fix queue lock name.
- Fix syntax error.
- Rework bypassing of task queue lock.
- Fix lock waiting for tests.
- Use Drupal API function for lock waiting.
- #2479345: Wait for a lock on the tasks queue to be available.
- #2479885: Remove anchors and double-escaping from regexes.
- #2479885: Fix mysql filtering regex anchors.
- Add provision-tests-new-run command to start testing D8.
- #2474197: Debian package for aegir3 should depend on curl
- #2475527: Catch additional DB connection failures, and provide more meaningful error messages.
- Generate an SSL certificate for use in the default SSL vhost.
- Add NameVirtualHost for SSL, regardless of IP addresses.
- Merge branch 'dev/mysqldump-error-check' into 7.x-3.x
- Fix up docs.
- Anchor regexes, cleanup docs.
- Split regexes into helper function with alter hook and static caching, and add docs for new hook.
- Clarify docs.
- Split mysqldump filtering into a helper function.
- Dedupe generating descriptors fro proc_open().
- Document new method and fix typos.
- Dedupe temporary mysql config generation.
- Fix up some syntax errors.
- fix comment typo
- handle mysqldump errors again
- #2471805: Add hook_provision_deploy_options_alter() invocations to migrate and restore, and document it in te API.
- #2471805: Add hook to allow injecting options into clone deploy tasks.
- Rename incorrect verify method.
- #2468203: provision-save: default profile from 'default' to 'standard'
- upgrade.sh: move variables to the top
- upgrade.sh: move function definition
- ssue #2460023: Fix existing symlink problem in Ubuntu/Debian upgrades.
- Cleanup upgrade.sh script
- Rename DRUSH_VERSION variable to separate current and new
- Fix Drush version in upgrade script
Changes to hosting since 7.x-3.0-beta1:
- escape < and >
- D7 API update
- code style updates
- Form API update
- Allow sites to be installed when SSL is enabled, but no servers provide the SSL service yet.
- Fix typo in table class.
- Redirect to home page after deleting a Hosting object.
- Allow re-use of deleted servers' hostnames.
- Don't block tasks queue when running queue daemon.
- Release locks in other shotdown contexts.
- Move lock acquisition and release outside of main queue daemon loop.
- Fix queue lock name.
- #2479345: Refactor queue locking to use Drupal API.
- #2464949: Avoid needless saving of related nodes on context_import
- Fix codestyle
- Log which task starts processing
- Pass less optionss ... to prevent Unknown option errors.
- Stop overwriting node titles when importing context data.
- Fix platform migrate confirmation form status.
- #2479001 by mwisner: Fix platform migrate task form does not rebuild correctly
- Fix platform migrate form checks for sites and available target platforms.
- Provide a default platform.
- Allow platform paths to have slashes.
- #2469179: Remove duplicate call to hosting_ssl_filter_key, its in hosting_ssl_nodeapi_site_validate
- #2469179: Improve saving a new SSL key
- #2475455: Document the place for am intermediate certificate.
- Try again to keep warning/error jump links within overlay.
- Keep warning/error jump links within overlay.
- #2175619: Add links to first warning or error in task log.
- Fix check for a platform's path already existing.
- #2474801: Fix task cancellation.
- Syntax fix.
- Add global option to always, never or prompt to delete site backups.
- #1397798 by chertzog, Dane Powell, et. al.: Optionally delete site backups when a site is deleted.
- #2471094: Update the site node with the database name after imports.
- Import site by calling the relevant function directly instead of invoking a sub-process, etc.
- Revert "properly import db_name after site import"
- Merge branch 'dev/hosting-import-dbname' into 7.x-3.x
- partly revert e3bb221: properly save drush output
- properly import db_name after site import
- code cleanup
- Improve ssl description
- #2453807: Convert clients list block to views
- Merge backend output.
- #2470297: Run 'hosting-import' after 'provision-import' on 'import' tasks.
- Revert portion of last commit, included by mistake. It will be part of the nest commit.
- Show backend output that was being lost.
- #2470297: Stop overwriting context prior to import.
- Set platform path automatically, based on title.
- Move site SSL fields into fieldset.
- Merge remote-tracking branch 'origin/dev-sni' into 7.x-3.x
- #2069431: Indicate next steps after enabling the SSL feature.
- Fix backup-delete form validation warnings.
- Fix restore form validation warning.
- #2460937: Fix backup deletion.
- Add some checks to backup delete validation.
- only warn about SNI, do not disable SSL, when IP allocation fails
- do not set IP address if none provided, this should permit SNI, in the frontend at least
6. Known issues
- Drupal 8 support is (Chasing HEAD) not complete.
- Drush master has an issue on Debian Wheezy (https://www.drupal.org/node/2491941)
2.4 release notes
The Aegir team is proud to announce the forth release in the stable 2.x release branch!
This maintenance release ships with a security fix for Apache installations. Everyone is encouraged to upgrade. Re-verify is required for all sites (Views bulk operations makes this easy)
Also, worth noting is the simultaneous release of our second beta for Aegir 3. Debian packages are now being provided for Aegir 3, so trying it out is easier than ever!
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is at:
http://community.aegirproject.org/upgrading
Within those sections, you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
N.B. Issue #2146977: Broken backward compatibility with IP based vhosts may affect those upgrading from 1.x. It appears that running 'verify' tasks on all sites should resolve the issue.
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Known issues
Being really open about our project, we have never hidden the fact that some things, sometimes, do not work in Aegir. Our issue trackers are public, and we've made it a point of honor not only to document clearly what is wrong in our releases as soon as we find out about it, but also to reroll new releases when we fix it.
That being said, 2.4 still has a number of issues and design flaws. This is the list of all issues marked "major" in the queue right now. Most issues are now likely to be fixed in the 3.x development branch, and unlikely to be backported unless considered critical.
As mentioned in the previous section, Issue #2146977: Broken backward compatibility with IP based vhosts is still listed as a 'critical' issue against the 2.x branch. However, it should only affect those upgrading from 1.x, and has a fairly simple work-around. If you come across this behaviour during your upgrade, please post a comment to the issue, so we can confirm that it still exists.
4. Features
5. Bug fixes
- Just the mentioned security fix
1.12 release notes
The Aegir team is proud to announce the twelfth (and final!) release in the now deprecated 1.x release branch! This release was necessary to cover the new Modalframe security release that made 1.11 uninstallable. Everyone is encouraged to upgrade to the latest 2.x release.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Known issues
The modalframe upgrade broke popups, but the content is visible so it is a minor regression. Upgrade to 2.x to get a real fix.
Otherwise the same as the 1.11 release.
4. Features
5. Security fixes
- modal frame was updated to cover for SA-CONTRIB-2014-012 (#2191827)
6. Bug fixes
- the jQuery download URL was fixed
3.0 alphas, betas & release candidates
This section documents all the release candidates towards the 3.0 release.
3.0-alpha1 release notes
The Aegir team is pleased to announce the first preview release (alpha) of the Aegir 3.x branch, after months of development. We have been pretty busy with the maintenance of the 2.x branch, so development has been sometimes intermittent, but it is not over yet!
This release is only a first of a series of alpha releases that are tailored to make upcoming changes available to a wider audience than people ready to install from git, but also to make features more widely available and tested. More alpha releases will be published when there are substantial changes.
In the issue queues the central roadmap issue is: #1261030 [meta] Roadmap: Aegir 3.x (D7 port)
1. Major changes
- Frontend ported to Drupal 7
- Raised minimum Drush version to 6.x
- Drush 7.x is supported, but that's a matter of chasing head.
2. Major API changes
to be determined.
3. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
Note that you should upgrade to the latest 1.x release (currently: 1.9) before attempting the upgrade to 2.x. This is especially important if you are running a pre-1.0 release (poor you!).
It is still imperative that you read the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
4. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
5. Features and improvements
In addition to the "major changes" mentioned above, the following should also be noted.
6. Bugfixes
This is a non-exhaustive list.
7. Known issues
- The devel module is listed as dependancy for the hostmaster install profile, but is not included in the drupal.org build.
3.0-alpha2 release notes
The Aegir team is pleased to announce the second preview release (alpha) of the Aegir 3.x branch, after months of development. Development has been sometimes intermittent, but it is not over yet!
This release is tailored to make upcoming changes available to a wider audience than people ready to install from git, but also to make features more widely available and tested. More alpha releases will be published when there are substantial changes.
In the issue queues the central roadmap issue is: #1261030 [meta] Roadmap: Aegir 3.x (D7 port)
1. Major changes
- Improved Drupal 8 support
2. Major API changes
to be determined.
3. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
Note that you should upgrade to the latest 2.x release (currently: 2.1) before attempting the upgrade to 3.x..
It is still imperative that you read the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
4. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
5. Features and improvements
In addition to the "major changes" mentioned above, the following should also be noted.
6. Bugfixes
This is a non-exhaustive list.
Changes since 7.x-3.0-alpha1:
- Improve readability of message
- #2348173 by helmo: Remove unused vars causing trouble
- #2372653 by helmo: Add --no-autocommit when dumping MySQL tables
- typo
- Fix platform name in demo content
- don't make the example module warn so much
- propagate example_field properly to the backend
- Improve debug messages in Config classes.
- Nginx: Helper locations to avoid 404 on legacy images paths (subdir only)
- Nginx: Fix spacing
- Nginx: Add missing variables in subdirectory config template.
- don't allow d() to run before drush is fully initialized
- clarify little debugging markers
- silence the last warning, it's actually not one
- add debugging: when no service type is specified from the frontend, yell a warning
- remove dead code: $base_dir is not used anywhere in that function
- Split arrays and hooks to alter chmod and chgrp file operations.
- Add server data example module, which doesn't currently work.
- Add post_verify hook for site_data module.
- Remove legacy drush load for site data example module.
- revert previous commit, it's too noisy
- clarify traces in site_data module
- Nginx: Add the fix for known problem with files/imagecache in legacy D6 sites (again).
- Remove openatrium2 from testcase, build fails too often on external issues
- Update openatrium2 test platform to new tag
- Rename makefiles to .make extension
- #2373835 by helmo | realityloop: 3.x doesn't currently install.
- Nginx: Fail early if any required db credentials are empty, to never create broken vhost.
- Nginx: Block semalt botnet (extended boa mode only)
- Nginx: Batch merge updates from 6.x configuration (2)
- Nginx: Batch merge from 6.x configuration.
- Correct package name in changelog
- Update debian package info to have aegir3 in the name
- Fix branch name for 7.x-3.x in debian git-buildpackage
- #2358795: Adding a "$force" parameter to setProperty() to allow overwriting of default context properties.
- #2358795 Adding a provision-save to provision_verify command.
- #2361729 by joestewart: Fixed D8 Platform verify doesn't add module version to package.
- #2365171 by joestewart: Fixed D8 Contrib profiles in /profiles undetected.
- #2362707 by joestewart: D8: update private and temporary paths after cloning
- allow overrides to the site-specific drushrc.php file
- #2163979 - Check if field_info_field_map() is available to not break support for old D7 versions.
- #1995506 by Jon Pugh: Added Implement "--profile" option for "drush hostmaster-install".
- Add API doc for new hook.
- Allow altering the list of directories to recurse into on site/platform creation.
- Removed a bashism
- #2348879 by Deciphered, helmo: Fixed Welcome Mail - url() deprecated.
- Remove a trailing space char
- refactor
- Improve error handling, better message and fail earlier.
- #2300537 by helmo | ergonlogic: Added Merge Provision extensions into Hosting modules.
- Make sure that db_port is never empty and defaults to 3306.
- Add drupal 8 to provision-demo-content
- #2346903 by helmo: Added Always disable allow_authorize_operations.
- #2311005 by Deciphered, clemens.tolboom: Fixed D8 Site install fails on undefined indexes.
- #2306055 by Deciphered, realityloop, helmo, clemens.tolboom: Fixed Platform verify fails for Drupal 8 under Aegir 3.x.
- #2346967 by Deciphered: Fixed Onetime login link broken.
- Pin test makefile of openatrium to stable version, to avoid build problems.
7. Known issues
- Drupal 8 support is (Chasing HEAD) not complete.
- Upgrading a hosted site from D6 to D7 has complications with Drush > 7.0.0-alpha6 issue
3.0-beta1 release notes
The Aegir team is pleased to announce the third preview release (beta) of the Aegir 3.x branch, after months of development. Development has been sometimes intermittent, but it is not over yet!
This release is tailored to make upcoming changes available to a wider audience than people ready to install from git, but also to make features more widely available and tested. More alpha releases will be published when there are substantial changes.
In the issue queues the central roadmap issue is: #1261030 [meta] Roadmap: Aegir 3.x (D7 port)
1. Major changes
- Improved Drupal 8 support
- Both Drush stable (6.5.0) and 7.x (the master branch) are supported.
2. Major API changes
to be determined.
3. Installing and upgrading
The canonical source of installation documentation is
http://docs.aegirproject.org/en/latest/install.html
We're in the process of converting our docs, the old version is on http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
Note that you should upgrade to the latest 2.x release (currently: 2.3) before attempting the upgrade to 3.x..
It is still imperative that you read the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
4. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
5. Features and improvements
In addition to the "major changes" mentioned above, the following should also be noted.
Changes to hostmaster since 7.x-3.0-alpha2:
- #2450683 by cweagans: Enable r4032login in the default hostmaster install
- Update the ctools version in our makefile.
- Update the entity API version in our makefile.
- #2421341 by clemens.tolboom, helmo: Add default permissions on install
- Regular contrib updates for admin_menu, betterlogin and ctools
- Update the views version in our makefile.
- #2393567: Betterlogin's anon redirect causes issues with services
Changes to hosting since 7.x-3.0-alpha2:
- #2448855 by Pol: Makefile field not saved when importing platforms
- log task processing in drupal
- #2393579: Signup feature cleanup
- #2407885: refactor SSL forms - hide other SSL fields when not enabled.
- SSL code cleanup
- Rename hosting_ip_delete to _hosting_ip_delete to avoid looking like hook_delete
- Rename hosting_ip_load to _hosting_ip_load to avoid looking like hook_load
- Rename hosting_ip_validate to _hosting_ip_validate to avoid looking like hook_validate
- Rename hosting_ip_save to _hosting_ip_save to avoid looking like hook_save
- Rename hosting_ip_view to _hosting_ip_view to avoid looking like hook_view
- Fix typo in variable name
- Allow numbers in Hosting feature names.
- Fix logic flaw
- Fix remaining boolean in access grants.
- #2410603: Fix default value
- #2410603: Working copy isn't currently working
- Remove duplicate reaction to enabling modules.
- Allow scripts to skip automatic verify tasks when enabling modules by passing '--no-verify'.
- Hook into module being disabled
- node_revisions table name changed to node_revision with D7
- #2430807 by clemens.tolboom, helmo: Roles order is reversed to advised order.
- Fix default permissions for optional submodules
- Fix typo in permission name
- #2432397: Add static cache to hook_node_grants
- #2430809 by clemens.tolboom: Notice: Undefined variable: grants
- Do not add validation when the form elements are not added
- #2430103 by clemens.tolboom: hosting_client: Strict warning: Only variables should be passed by reference
- Update db placeholders
- #2410209: Use bigint to store backup size
- Log which drush task hooks we're invoking.
- #2210813 by ergonlogic, realityloop: Follow-up from D7 DBTNG changes
- Rename constant MAX_GROUP_LENGTH to HOSTING_CLIENT_MAX_GROUP_LENGTH
- #2408365 by clemens.tolboom: Strict warning when adding new client
Changes to provision since 7.x-3.0-alpha2:
- #2457359: Stop overriding Nginx SSL settings
- #2396299 by ergonlogic: Stop comparing platform package versions when said package is installed in the site
- Merge branch '7.x-3.x' of git.drupal.org:project/provision into 7.x-3.x
- Nginx: Stop the POST flood to /autodiscover/autodiscover.xml generated by MS Office/Outlook
- Debian: improve setting directory permissions.
- #2193427 by helmo: Remove Debian package requirement for Drush
- Make php5-gd a dependency of the aegir3-hostmaster package, as its required to install core
- Log site installation exception.
- Update absolute URLs to files for sites cloned/migrated/renamed
- Nginx: Use dummy db fastcgi_param placeholders if any of them is empty
- #2350695 by omega8cc, helmo: Profile is registered twice, also as a module, which causes warning
- Nginx: Remove webform keyword from regex locations - fixes #599
- Remove Drupal 8 specific stuff from D7/D6 template.
- Both maintenance_mode and clean_url are ignored in Drupal 8
- Fix for custom cpuinfo logic.
- Nginx: Sync configuration improvements.
- Remove docs for non-existing option
- Add the working-copy option to hostmaster-migrate, similar to hostmaster-install
- hostmaster-resume was renamed to hosting-resume
- #2421543 by notzach: Changing how Apache ServerAliases are defined
- Drupal 8 requires trusted_host_patterns defined in settings.php
- Drupal 8 requires container_yamls defined
- Nginx: Use safe fallback for mysteriously empty $db_port
- Nginx: Use reliable source for db_port to write in the vhost
- #2409085 by clemens.tolboom: Use mutatable interface in install_8.inc
- #2387953: Remove drush-make from Debian suggest line.
- #2276557 by zxaos, bgm: allow aegir2-provision package to accept mariadb or mysql
- Nginx: Fix for D8-specific /cron/ location caching (extended mode only).
- Nginx: Fix for D8-specific /cron/ location regex.
- Update deploy_8.inc to sync it with improvements in deploy_7.inc
- Set files paths on D8 install to avoid using system default /tmp if path.temporary is not defined.
- Nginx: Sync headers configuration in the extended mode.
- Do not copy CDN aliases to the cloned site --CDN vhost.
- Detect and update default D8 site name, if needed.
- #2362703 by joestewart - import_8 ineffective.
- Sync with 6.x workaround for systems with access to /data/all/cpuinfo instead of /proc/cpuinfo
- Sync subdirectory support.
- Force clean URLs for Drupal 8.
- The /login suffix is no longer supported in Drupal 8 and results with confusing 404 error.
- Avoid broken install on D8 core where sites/all doesn't exist by default.
- #2183323 - Remove all remnants of $GLOBALS['conf']
- Fix for removed drupal_mail() which breaks Drupal 8 head install if used.
- Restore missing !edit_uri with hardcoded standard URI.
- Nginx: Drupal 8 with clean urls enabled should use /cron/ URI.
- Nginx: Sync config templates.
- Add missing code for subdirectory mode detection and support.
- Detect if the platform stays the same only when site name changes.
- Use $new_name instead of $new_uri to avoid confusion.
- Compare $new_uri with d()->name and not d()->uri in the Site Rename Check.
6. Known issues
- Drupal 8 support is (Chasing HEAD) not complete.
- The upgrade.sh.txt in provision still had Drush 5.10.0, be sure to change it to 6.5.0 before using that script.
- The upgrade code in the Debian package could fail on an exsiting /usr/local/bin/drush file, this has been fixed in https://www.drupal.org/node/2460023
Older 2.x releases
This section documents all the older releases in the 2.x release cycle.
2.1 release notes
The Aegir team is proud to announce the second release in the stable 2.x release branch!
But first, here's a word from Antoine Beaupré:
This latest release of the Aegir project will be the last one for me as a release manager, as I will be stepping down from the leadership position I have filled in the 7 years of involvement in this amazing project. I will continue to contribute to the project, to a lesser extent. I will remain available, through Koumbit, to provide high-level consulting work for Aegir. We can also now welcome Praxis Labs Coop as a new consulting partner of the Aegir project. Praxis will help to finance ongoing Aegir development by subsidizing part of my salary, in a similar fashion to how Omega8.cc had in the past.
Working with the Aegir project was a huge challenge, both from a technical perspective, but also from a human standpoint, as it is a large and complex project that may seem daunting for new users. But with regular releases and continuous support in the issue queue, I think we have been able to foster a great community of contributors that goes beyond what a single person can do, as Adrian proved when he left the project with the 1.0 release in 2011. Christopher (ergonlogic) and other contributors have shown they are in a good position to take over that leadership role, and will do so from now on.
Now it is my turn to move on, and I am quite proud of what I leave behind. This project allowed me to travel around the world, meet incredible people and learn a lot about hosting and provisioning. It is however time for me to move on. I have been involved with the Drupal community for over 13 years, and things, to put it lightly, have changed. PHP is attempting to become a real programming language, something that I didn't believe could or should happen, and I wish to move on to other paradigms. Python and Haskell are waiting for me around this turn in the road, and even Perl and shell scripting look awesome in comparison to those dreaded PHP years. Besides, since Aegir started, lots of configuration management systems have popped up that we can leverage to host way more than PHP and Drupal: Docker, Chef, Puppet hold lots of promise for those of us who dare to get out of the NIH syndrome. Hopefully, that energy will be reusable in the Aegir project as we head towards Aegir 4!
And so, thank you, Antoine, for all your great contributions as leader of the Aegir project! You have been an inspiration to many of us, and have provided superbly effective technical leadership.
This release ships with support for Apache 2.4, along with updates to Drupal core and ModalframeAPI. A number of bug-fixes are included affecting deployment of Drupal 7 sites, aliases with sub-directory installs, backups, among other things. See below for the full lists of new features and bug fixes. Everyone is encouraged to upgrade.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is at:
http://community.aegirproject.org/upgrading
Within those sections, you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. Noone likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Known issues
Being really open about our project, we have never hidden the fact that some things, sometimes, do not work in Aegir. Our issue trackers are public, and we've made it a point of honor not only to document clearly what is wrong in our releases as soon as we find out about it, but also to reroll new releases when we fix it.
That being said, 2.1 still has a number of issues and design flaws. This is the list of all issues marked "major" in the queue right now. Most issues are now likely to be fixed in the 3.x development branch, and unlikely to be backported unless considered critical.
hosting-queued
doesn't get properly re-enabled on upgrades, see #2114675 for a workaround.- previous Debian packages may remove the
aegir.conf
symlink before being upgraded. this is fixed in 2.0 but will occur (it's inevitable) during upgrades from 1.x releases. details and simple workaround in #2121263
4. Features
- Apache 2.4 is now supported in the 1.x and 2.x branches, see #2153929 and #2155445
- updated core to Drupal 6.31
- #2191827 by Jon Pugh: Update modalframe
5. Bug fixes
- #2157785 by helmo: Deploy_7 code lacks functionality from D6 code
- #2169025 by gboudrias: Fix non-gzipped backups can't be restored
- #2213387 by Chris Moates: Fix alias handling when using subdirs support
- Cleanup some file permissions from 755 to 644
- #2213787 by SocialNicheGuru, helmo: Undefined property: stdClass::$ssl_key_new hosting_ssl.nodeapi.inc:252
- Fixed reference in comment and cleanup notices
- #2171075 by helmo: Fix hostmaster-resume command options. (3 months ago)
- #2189687 Redirect to site-alias not working
2.0 release notes
The Aegir team is very pleased to announce the official release of Aegir 2.0.0. This long-awaited release ships with significant stability improvements, Drush 5 and 6 support, subdirectory multisite support, improved nginx support, native views and much, much more! We unfortunately had to drop support for Drupal 5, as Drush 5 and 6 dropped support for this unsupported Drupal release.
This release deprecates the 1.x branch, marking the 1.11 release as the last one of the 1.x branch. All Aegir users are strongly encouraged to upgrade to the 2.0 release. The upgrade has been thoroughly tested and works fairly reliably, except some documented minor issues.
We have a very dynamic community of contrib developers, and various projects built atop Aegir. So if you were waiting for Aegir 2 to be stable, easy to install, production-ready, now is the time. The 2.x branch will be well supported as a lot of shops are running it in production already.
1. Major changes in this release
Since th 1.0 stable release (1.0) in April 2011, we've done an incredible amount of work. The code size nearly doubled as did the development team and user base. We have accomplished a significant number of the release goals we set for 2.0, but not all. Here is a broad breakdown of the most important improvements between the 1.11 and 2.0 releases:
- Subdirectory multisite support
- you can now have different sites in example.com/foo and example.com/bar, a great feature if you do not control DNS, as well as for local development!
- SSL improvements
- IP allocation improvement: addresses can be changed, added and removed more easily, and are managed from the front-end now.
- SSL certificate generation now has saner defaults (e.g. 2048bits)
- better error handling
- Drush 5 & 6 support
- includes better support for archive-dump command
- means we also drop the dependency on drush make 2.3, now included in Drush 5
- support for Drush 4 has been dropped, the minimum version is now 5.5, but Drush 6 or 5.10+ is recommended.
- Packaging improvements and changes
- Nginx support in the Debian package
- now a "native" debian package
- package name changed: it is now
aegir2
,aegir2-provision
, etc, to avoid overwriting the previous package - note that aegir 1 and 2 can not be installed in parallel - hosting and eldir now split back in their own projects on dripal.org
- using Drupal.org packaging tools to provide a full tarball for the whole frontend, speeding up installs significantly
- Nginx improvements:
- fixed support for nginx 1.3 and newer
- better defaults for caching
- SSL and Nginx are now officially supported (not marked as "experimental")
- New modules:
- the
hosting-queue-runner
module is now merged into core ashosting-queued
- "pack" module, designed as a lightweight replacement to the "cluster" module, now available as an experimental extension
- the
- Hosting-queued improvements
- run with a lower priority ("niced")
- improved portability and reliability of startup script
- enabled by default in the Debian package
- improved stability
- Code refactoring and improvements:
- now using the Symphony autoloading code
- fix coding style in a lot of source files
- improved builtin test suite
- Frontend improvements:
- all custom displays ported to Views and VBO
- nicer frontpage
- new and improved roles with more useful default permission sets
- Documentation improvements:
- Our documentation initiative to better document the Hostmaster API has progressed nicely
- we'll be launching an updated version of our API site (along the lines of api.drupal.org) shortly, to ease exploring the code-base of the Aegir Project, see this issue.
For a more detailed list of new functionalities, bugfixes and API changes, see below.
Some goals of the original 2.0 roadmap have not been accomplished, namely:
- modular backup, platform and queuing systems - although work has started on Wordpress support, and an overhaul of the Aegir architecture is considered for 4.0, which will affect queuing systems (see below)
- DNS, PostgreSQL and statistics efforts have mostly stalled
- clone and migrate optimizations still have to be completed
- completely automatic upgrades have not been implemented, although the Debian package upgrades are working generally well
- the release was about 2 months late from the october 31st estimate
1.1. 2.x branch maintenance policy
With the 2.0 release, the Aegir project wished to adopt the growing standard of Semantic Versioning. Unfortunately, the tools on Drupal.org do not allow 3-digits version numbers in contrib, something of a blocker to allow us to use that great numbering scheme. So we will stick to our previous scheme for now, but we'll consider a switch to semver for 3.0.0.
No API change will be done in the 2.x branch. All core development will now happen on the 3.x branch, which shifts the focus of development to the port of the frontend to Drupal 7 (almost complete).
Note that the 2.x API has already been frozen for some time, since the first release candidate, to be more precise. Major changes to 2.x will not be committed unless they are first tested in 3.x and merged back. However, we wish the 2.x branch to be relatively short-lived, and it is more likely that major changes will not be backported from 3.x unless absolutely necessary for users needing a Drupal 6 Aegir frontend.
The 1.x branch is now deprecated and the 1.11 release will be the last release of that venerable branch, unless we need another release to fix the upgrade path to the 2.x branch. This is considered very unlikely. Security fixes or critical issues will therefore not be backported to the 1.x branch.
1.2. The future
While we're very proud of what we've accomplished in Aegir 2.0, we've also been working in parallel to port Aegir to Drupal 7. Aegir 3.0 has already begun as a fairly straight-forward port. As such, we'll be releasing our first alpha of Aegir 3 shortly. We would like to have a stable Aegir 3.0 release before the release of Drupal 8, to allow for users to transition as Drupal 6 enters its end-of-life phase.
The reason for the change to semantic versioning is that we've been discussing a full re-write of Aegir for some time, and would like to do so with Aegir 4. The changes are likely to be fairly drastic, and so we want to be able to keep moving Aegir 3 forward in the mean time. By moving to Drupal 7, we expect to have bought ourselves 2+ years of breathing room within which to accomplish this re-architecture and the semantic versioning system allows us to integrate major changes in the 3.x branch progressively.
When the Aegir project started (back in 2008), the free software options for systems management software were very limited. As a result, we wrote our own code to do things like deploying code, writing configuration files, starting and stopping services, and so forth. Now, as 2014 begins, there are a wealth of tools that perform these functions (e.g. Puppet and Chef, but also Openshift, Docker, etc), each with their own communities supporting them. We are exploring how we might be able to leverage these tools, rather than maintaining our own partial implementations, that will never likely be as robust as these more specialized projects.
We feel that the time is opportune since Drupal 8 will require significant re-writing of the front-end components. In addition, re-writing the backend so completely will allow us to seriously consider moving to a programming language other than PHP; one that would be better suited to the project's long-term goals, such as Python or Ruby.
2. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
For users coming from the 2.0 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
Note that you should upgrade to the latest 1.x release (currently: 1.9) before attempting the upgrade to 2.x. This is especially important if you are running a pre-1.0 release (poor you!).
2.1. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
2.2. Known issues
Being really open about our project, we have never hidden the fact that some things, sometimes, do not work in Aegir. Our issue trackers are public, and we've made it a point of honor not only to document clearly what is wrong in our releases as soon as we find out about it, but also to reroll new releases when we fix it.
That being said, 2.0 still has a number of issues and design flaws. This is the list of all issues marked "major" in the queue right now. Most issues are now likely to be fixed in the 3.x development branch, and unlikely to be backported unless considered critical.
hosting-queued
doesn't get properly re-enabled on upgrades, see #2114675 for a workaround.- Ubuntu saucy has some changes in apache config wich breaks Aegir #2155705
- previous Debian packages may remove the
aegir.conf
symlink before being upgraded. this is fixed in 2.0 but will occur (it's inevitable) during upgrade. details and simple workaround in #2121263
3. Complete list of changes
Her is the complete list of changes, since the 1.x branch, some of which were already mentioned in previous release notes for alpha/beta/rc releases.
3.1. API changes
Those changes are, as usual, more explicitly documented in the upgrade path documentation.
- hosting-task now needs a
--force
argument to run a non-queued task - the '--force' option also allows running tasks that appear to be running in the queue
- functions that were deprecated in 1.x are now removed
- the
email
andclient_email
database fields are now removed from client and site node types - numerous changes to the IP allocation and SSL management code
- deprecate hosting_ip_delete_revision(), dupe of hosting_ip_delete() now that revisions are gone (8 weeks ago)
- remove deprecated DEBUG flag in debian package
- #1785624: Some Drupal API changes in D7 (and D8) are not used/respected properly
- #1945950: Rename provision_drupal_sync_site_back()
- #1083366: Make the spokes authoritative for files/ and private/ directories
- #1812338: Refactor sync back
- a new control file has been introduced in /etc/nginx/basic_nginx.conf to force the nginx configuration to be the "simpler" one (see #1635596)
- #1987026: Move generated platform drushrc.php to sites/all/drush
- Multiple files can be managed in a single context now (see issue #2000038, issue #1784108)
- #1034520: Cleanup package instances when deleting sites and platforms.
- #1830220: Drop support for Drupal 5.
- #1975086: Move log parsing and status updates to seperate functions and call them from a shutdown function.
- Pass the entity type when we're sync'ing package instances.
- Save the platform field when creating a package instance record.
- Move to individual operation callbacks for VBO tasks.
- #2022849: Record disable and delete backups in the database.
- #2031491: Rename SSL permission to be more descriptive.
- Fix node grants so that proper permissions are respected on all Aegir node types.
- Add 'administer' permissions for platforms and servers, and allow platforms to be viewed.
- #1283738: Allow other commands to add or alter the directories to be created.
- remove version pinning in hostmaster, our release process now again needs to modify only one makefile (#2002114)
- #1986928: Provide 2.x upgrade guidance for services
- #2099889: More hosting_features checks in _drush_load hooks?
- #1882708: Unused 'release_id' field in 'hosting_platform'
- #2012508: hosting_context_name returns '@' if node not found
- #1283738: Add new hook provision_drupal_create_directories in _provision_drupal_create_directories
- There is now a registry in the backend of the features enabled in the front-end (in /var/aegir/.drush/drushrc.php) which allows backend components to better determine if they should act
- If
/var/aegir
is a symlink, it will be destroyed by the debian package, make sure the Aegir $HOME is properly set to work around that, see issue #2118857
3.2. New features
In addition to the "major changes" mentioned above, the following should also be noted.
- #1830994: Login directly via 'login' link
- #1853588: Sort Install profiles on Site add form
- #709862: Allow drush modules to add to apache vhost config template
- #1861896: Make $type argument of function hosting_available_tasks() optional.
- #1843198: Add Content top and bottom to Eldir.
- #1826074: Add tests to ensure site directories are migrated
- #1464220: Support alternative 'Hostmaster' profiles
- #610040: Add X links on listing pages
- #710834: notify clients on site migrations
- #1260066: Allow use of the working copy option when creating a platform
- #1047992: exportable backups
- #1189556: Pull the queue runner into Aegir 2
- #1408410: Add microcaching to Nginx config by default
- #1138104: Tiny module to automatically fill in platform publish path based on title
- #1929372: Flag tasks with logged warnings
- #1515416: On a package page, show table listing sites and platforms using the package
- #1853620: Add db_name to site summary
- #1681904: Ability to configure a url to redirect to in site configuration.
- #1968226: manage each IP individually on the server level
- add uninstall command
- Nginx Security: BEAST attack protection and fix for PCI compliance.
- #1980136: Allow setting default profile from the front-end.
- #588728: Replace custom lists with Views and VBO for sites, platforms and packages.
- Support for installing in subdirectories (issue #705026)
- Proper homepage on startup so non-logged-in users don't see an error page, and new users have basic instructions (issue #1793740)
- SSL support for clusters should now work properly (issue #2000964)
- New roles added: 'aegir platform manager', 'aegir administrator' (issue #1403208)
- Convert list of platforms to use views (Issue #1876350)
- #600350: provide a nice "history" of ran tasks
- #1988642: Add site by platform
- #1912238: automatically build and upload aegir 2.x packages
- #585830: usability story: "How do I create a client?"
- #1212086: Platforms should be locked by default
- #1403208: Add additional roles
- #1174440: provide a way to show site's DB username/password in the frontend
- #606120: Disabled sites should be class hosting-warning, enabled but problematic sites should be hosting-error
- #905326: Improve file path changes.
- #1201174: Make UID1 username configurable.
- #1345118: Make platform access control an autocomplete form.
- #1515416: Replace listing of sites and platforms using a package with a view.
- #1975086: Add 'update status' button to tasks.
- #2005310: Add VBO operations to platforms view.
- #2006074: Enable backups via VBO.
- #2025787: Open site goto link in a new window/tab.
- #2022813: Expose backups to Views.
- #2027269: Update a task's status after all Drush operations are complete.
- #2031491: Review and update permissions for all roles.
- #2031765: Clean up VBO operations with Action Permissions.
- #2035873: Show package 'popularity' on platform package view
- #2036283: Make Aegir aware of site-specific packages.
- #2036793 Ignore hidden modules and profiles.
- #2037045: Change some log statuses to better match Drush's log styling.
- #2037965: Clean up hosting-pause.
- #2038279: Validate email during site install.
- Display all validation errors when adding a client, and limit to 20 suggestions.
- Flag rollbacks as warnings in our task logs.
- Handle package page views separately for sites and platforms.
- Update default hosting_site views to use new access plugin for blocks, and add a couple more displays for non admin listings.
- Add Views access plugin for hosting_site and hosting_package.
- Add views handler to filter packages by status.
- #2069387: Support nonstandard ports on hostmaster-install command.
- #2067617: Allow hostmaster-install to accept '--working-copy' option.
- Allow the redirect to the welcome page to be turned off.
- aegir install time reduced by about 60 seconds (on a cable uplink) by using Drupal.org distributions (#2002114)
- #2001964: Upgrade Views Bulk Operations 6.x-1.16
- #2099057: Allow _provision_drupal_create_directories to skip chmod
- #2100251: Implement post-enable hook for 'hosting_*' modules
- #2020075: Refactor subdirs to use proper Config class(es)
- #2020079: Multi-server subdirectory support
- #2096629: hook_provision_drupal_create_directories_alter by ref
- #2086009: Logo Cannot be changed / disabled
- #2069387: Support nonstandard ports on hostmaster-install command
- #2074681: Ports are hardcoded in hostmaster.profile
- #2103173: Provide an example of saving data to a site context
3.3. Bugfixes
This list only shows bug fixes since the last release candidate (RC5), since the list of all bugfixes in Aegir 2 is truly enormous. Please review the individual release notes for more details, if you are curious.
- Improve the subdir related description in the site add/edit forms, to explain the most important details clearly enough.
- Do not allow duplicate subdir aliases - use hosting_alias_allow_domain() in hosting_alias_validate_subdir() directly.
- Update the inline how-to for sites in subdirs.
- #2020089 by ergonlogic - Allow example.com and example.com/foo domains (Hosting)
- #2151475 by helmo: Extra comment.
- #2151371 by helmo: Allow custom platform tasks.
- security fix: SA-CORE-2013-003, fix files/ protection
- #1220062 fix mild security issue with login-reset links.
- fix SSL certificate garbage collection, it was simply not working
- reformat complex query to be more readable
- also free SSL certificate when SSL is disabled on a site
- check only client nodes for duplicates, closes #2123355
- #2119101: Fix tasks are not sorted properly.
- don't verify deleted sites, fixes #2119111
- properly populate the SSL cert / IP association map
- properly count the aegir hosting aliases, fixes #2118917
- Fix malformed @defgroup.
- #2085077 by chertzog | ergonlogic: Fixed Invalid argument supplied for foreach() hosting.ip.inc:39.
- #2126895 by helmo: Upgrade.sh not compatible with Drush 6.
- #2020089 by ergonlogic - Allow example.com and example.com/foo domains (improved to fix a few dangerous errors) plus comments.
- Nginx: Use text/xml mime type for .xml URLs to restore defaults.
- Fix the check for parent site existence - we should look for drush alias file and not a vhost (which can be dummy subdir vhost).
- Nginx: Fix aliases in redirects on the fly.
- Nginx: Convert subdir alias name properly when used as server_name in redirection.
- #2020089 by ergonlogic - Allow example.com and example.com/foo domains (Apache)
- Nginx: Redirect to working homepage for subdir based site.
- #2100181 by ergonlogic - Symlink missing for D6 subdir sites.
- #2146711 by helmo: PHP 5.3 compatible (Nginx).
- #2038891 by drastik: add missing --client-email option
- #2020091 by ergonlogic - Support subdirs with Nginx.
- fix redirection for non-ssl hosts, issue #2148671 by logaritmisk
- #2157785 by helmo: Remove unused variables, these get written to the site's settings.php anyway.
- #2157783 by helmo: Deploy_7 code has an unused old_uri.
- don't go around destroying apache configs on upgrades or removal
- get the aegir home directory dynamically
- rename VARLIB variable to AEGIRHOME for clarity
- fix drush dependency: we need 5.10 to treat hostmaster as core in drush make
- Further fixes for strict options checking in Drush 6.
- #2135999: Fix for strict options checking in Drush 6.
- Update site_data example extension with new load method.
- Fix naming of hook in api docs.
- be more tolerant about the way the alias gets passed from the frontend
- #2117227 by Ogredude - Missing newline causes syntax errors in vhost.d nginx config when redirects are enabled.
- Copy all makefiles into Debian package.
- Conditinally add subdirs/ to the debian package.
- Fix .deb dependency.
- #2113463: Fix take hosting_platform_pathauto out of experimental.
- #1376466 by Dane Powell: Fixed path not generated when copy/paste is used.
- #1898252 by Dane Powell: Fixed Edit option displayed on existing platforms.
3.4. Other issues
- #1923552: Rename 'Queue runner settings' tab to 'Queue daemon'
- #1834036: Add 'hosting platform pathauto' to the .gitignore
- #1785624: Some Drupal API changes in D7 (and D8) are not used/respected properly
- #1979496: Update upgrade.sh.txt for Drush 5
- #1974752: Document old_uri option for provision-deploy
2.2 release notes
2.2 was an abortive release, due to missing updates to contrib modules. Use 2.3 instead.
2.3 release notes
The Aegir team is proud to announce the third release in the stable 2.x release branch!
This maintenance release ships with a huge number of bug fixes since our last stable release, almost a whole year ago. There are also some security improvements for nginx users. See below for the full list of bug fixes. Everyone is encouraged to upgrade.
Also, worth noting is the simultaneous release of our first beta for Aegir 3. Debian packages are now being provided for Aegir 3, so trying it out is easier than ever!
N.B. The 2.2 release was invalid, as the contrib modules included in the drupal-org.make had not been updated.
1. Security improvements for nginx
Previously, Aegir stored SSL cipher and protocol settings per site in Nginx virtualhost configuration files. This included the enabling of SSLv3, which is vulnerable to POODLE attacks [1].
Because these settings were stored via templates, permanently removing SSLv3 support on Aegir-based Nginx SSL environments was not possible. These settings have now been removed from the templates that generate Nginx virtualhost files under Aegir. If you wish to either re-enable SSLv3, or otherwise alter SSL cipher and protocol settings in Nginx, we recommend you do so in the http {} context (e.g, in /etc/nginx/nginx.conf), which are applied globally and are never manipulated by Aegir.
Sites that were running on Nginx and had SSL enabled, should be re-verified to remove those settings.
Apache users (including those using SSL) were not affected.
[1] http://en.wikipedia.org/wiki/POODLE
2. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is at:
http://community.aegirproject.org/upgrading
Within those sections, you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
N.B. Issue #2146977: Broken backward compatibility with IP based vhosts may affect those upgrading from 1.x. It appears that running 'verify' tasks on all sites should resolve the issue.
3. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
4. Known issues
Being really open about our project, we have never hidden the fact that some things, sometimes, do not work in Aegir. Our issue trackers are public, and we've made it a point of honor not only to document clearly what is wrong in our releases as soon as we find out about it, but also to reroll new releases when we fix it.
That being said, 2.2 still has a number of issues and design flaws. This is the list of all issues marked "major" in the queue right now. Most issues are now likely to be fixed in the 3.x development branch, and unlikely to be backported unless considered critical.
As mentioned in the previous section, Issue #2146977: Broken backward compatibility with IP based vhosts is still listed as a 'critical' issue against the 2.x branch. However, it should only affect those upgrading from 1.x, and has a fairly simple work-around. If you come across this behaviour during your upgrade, please post a comment to the issue, so we can confirm that it still exists.
5. Features
#2267057: Pre-upgrade add ctools to 6.x-2.x
6. Bug fixes
Changes to hosting since 6.x-2.1:
- Make it clear that subdomain (foo) and subdirectory name (foo) must be identical.
- #2333911 - Restore the pre-views order of tasks in the queue blocks/lists.
- #2313327 by helmo | tvl: Fixed Unknown options for provision-verify.
- #2163525 by helmo | daften: Fixed Aegir doesn't delete old alias on migration of site.
Changes to provision since 6.x-2.1:
- #2457359: Stop overriding Nginx SSL settings
- Nginx: Stop the POST flood to /autodiscover/autodiscover.xml generated by MS Office/Outlook
- Nginx: Use dummy db fastcgi_param placeholders if any of them is empty
- #2350695 by omega8cc, helmo: Profile is registered twice, also as a module, which causes warning
- Nginx: Remove webform keyword from regex locations - fixes #599
- Nginx: Sync configuration improvements.
- Fix for custom cpuinfo logic.
- #2421543 by notzach: Changing how Apache ServerAliases are defined
- Nginx: Use safe fallback for mysteriously empty $db_port
- Nginx: Use reliable source for db_port to write in the vhost
- #2276557 by zxaos, bgm: allow aegir2-provision package to accept mariadb or mysql
- Nginx: Fix for D8-specific /cron/ location caching (extended mode only).
- Nginx: Fix for D8-specific /cron/ location regex.
- Nginx: Drupal 8 with clean urls enabled should use /cron/ URI.
- Nginx: Sync config templates.
- Detect if the platform stays the same only when site name changes.
- Standardize inline comments.
- #2372653 by helmo: Add --no-autocommit when dumping MySQL tables
- Use $new_name instead of $new_uri to avoid confusion.
- Compare $new_uri with d()->name and not d()->uri in the Site Rename Check.
- Nginx: Helper locations to avoid 404 on legacy images paths (subdir only)
- Nginx: Fix spacing
- Nginx: Add missing variables in subdirectory config template.
- don't allow d() to run before drush is fully initialized
- Nginx: Add the fix for known problem with files/imagecache in legacy D6 sites (again).
- Nginx: Fail early if any required db credentials are empty, to never create broken vhost.
- Nginx: Block semalt botnet (extended boa mode only)
- #2358977 by mrP - [nginx] Aegir redirection to non-install url leads to sites/$server_name/files 404 errors (sub-dir config sync)
- #2373923 by griz - https redirect problem with Nginx (fix tested)
- Nginx: Avoid redirect loops (really fixed).
- Nginx: Avoid redirect loops (fixed).
- Nginx: Avoid redirect loops.
- #2358977 by mrP - [nginx] Aegir redirection to non-install url leads to sites/$server_name/files 404 errors.
- Nginx: Simplify imagecache/styles support.
- Nginx: Remember real site name in $main_site_name and MAIN_SITE_NAME.
- Sync new lines.
- Nginx: proper sync with Apache redirects.
- Revert "Issue #2373923 by griz - https redirect problem with Nginx"
- #2373923 by griz - https redirect problem with Nginx
- Merge remote-tracking branch 'origin/6.x-2.x-backports' into 6.x-2.x
- #2163979 - Check if field_info_field_map() is available to not break support for old D7 versions.
- Make sure that db_port is never empty and defaults to 3306.
- #2266997 by helmo, cweagans: Added Do not automatically enable update module when installing a site.
- Nginx: Update vhosts templates to match BOA improvements #unforkboa
- Nginx: Sanitize aliases in vhost_disabled.tpl.php to avoid: 'nginx: [warn] server name "foo.com/bar" has suspicious symbols'
- Fix spacing in config lines.
- Nginx: Update config includes to match optional BOA features improvements #unforkboa
- Manage extra GRANTS to allow SQL remote access via SSH tunneling which depends on '127.0.0.1' and will not work with GRANTS for 'localhost'.
- Add support for file generated from /proc/cpuinfo on system with no access to /proc #unforkboa
- Ignore paths from OS X
- Backport provision_hosting_feature_enabled (2)
- Backport provision_hosting_feature_enabled()
- Fix typo.
- Shorten and simplify the subdirs checks code.
- Remove legacy subdir code and update checks.
- Remove whitespace.
- Use is_readable() instead of file_exists() when checking alias existence.
- Use is_dir() instead of file_exists() when checking directory existence.
- Use is_readable() check instead of insufficient file_exists() for config includes.
- Remove redundant file_exists() if is_readable() is also used.
- Add little debugging markers.
- Use strict checks: is_file() and/or is_link() instead of file_exists() before attempting unlink()
- Fix for mysterious warning "Could not create directory ." on Hostmaster site Verify.
- Nginx: Fix typo in vhosts templates.
- #2330781 - Use Drush native dt() wrapper instead of not always available t()
- #2329131 by dagomar: Fixed Uninstalled block module causes errors.
- Allow usage with Drush7, now that *.drush.load.inc's aren't being included.
- #2177315 by pwatzeels: Fixed Group permissions on private/temp folder not correct on remote server.
- #2296089 by ergonlogic, cosmicdreams: Fixed Installer can detect incorrect default web group.
- #2169287 by cableman0408: Fixed Setting user name twice (fails non-default installation profile).
- #2275467 by kristofferwiklund: Fixed incorrect variable in Debian postinstall script for Apache 2.4
- Stop false-positive warnings when SSL uses a wildcard.
- #2259461 by Liam McDermott - Remove too aggressive limit_conn directive in the Nginx config templates.
- Revert "change version information for release 2.1"
- #1168758 by helmo | acrollet: Optionally add --include-vcs argument when using drush rsync to sync remote platform.
- Extend provision_drupal_fetch_site with a parameter to specify site to fetch.
2.0 alphas, betas & release candidates
This section documents all the release candidates towards the 2.0 release.
2.0-rc5 release notes
The Aegir team is is pleased to announce the fifth release candidate for the upcoming Aegir 2.0 release!
This release ships a ton of bugfixes we have found in RC4. We also finalised subdirectory support which, while it still has some issues, is now actually working correctly, even on multiple servers. The IP allocation code for SSL on multiple servers was also fixed, making this probably the most solid multi-server Aegir release ever. We also did some API cleanups that seemed necessary before the final 2.0 release.
Finally, we should also mentionned that the install process was significantly sped up thanks to the use of Drupal.org distributions to ship a single tarball for the hostmaster platform instead of using a makefile to build it from its parts all the times. During out tests, the platform make
step went from 60 seconds to 1.5 seconds, an amazing improvement!
Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes, with 2.x
-specific instructions outlined in bold.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. You should be able to upgrade to 2.x releases from any release in the 1.x series, but it's better to upgrade to the latest 1.x release first.
Known issues
Those are the critical bugs that were found in this release that you may encounter and should watch out for.
- There was a problem in the Debian package, the original uploads were missing some critical files (subdir support and some makefiles). This was fixed in an intermediate package named
2.0~rc5.1
. - If
/var/aegir
is a symlink, it will be destroyed, see issue #2118857 for a workaround. hosting-queued
doesn't get properly re-enabled on upgrades, see #2114675 for a workaround.- Some upgrades fail with mysterious redirection failures, see #2118061 for details.
- Some upgrades may loop over one of the update_N() functions, see #2118917 for details and workaround.
- Upgrading through Debian packages removes the
aegir.conf
Apache configuration file, dropping all sites, during the upgrade, see #2121263. - Ubuntu sausy has some changes in apache config wich breaks Aegir #2155705
Detailed list of changes
API CHANGES
- remove version pinning in hostmaster, our release process now again needs to modify only one makefile (#2002114)
- #1986928: Provide 2.x upgrade guidance for services
- #2099889: More hosting_features checks in _drush_load hooks?
- #1882708: Unused 'release_id' field in 'hosting_platform'
- #2012508: hosting_context_name returns '@' if node not found
- #1283738: Add new hook provision_drupal_create_directories in _provision_drupal_create_directories
- the package name changed (for example) from aegir-hostmaster2 to aegir2-hostmaster, compatibility shims are available for people having installed aegir2 before, but they will be dropped in 2.0, so make sure you upgrade to this release before upgrading to aegir 2.0
NEW FEATURES
- aegir install time reduced by about 60 seconds (on a cable uplink) by using Drupal.org distributions (#2002114)
- #2001964: Upgrade Views Bulk Operations 6.x-1.16
- #2099057: Allow _provision_drupal_create_directories to skip chmod
- #2100251: Implement post-enable hook for 'hosting_*' modules
- #2020075: Refactor subdirs to use proper Config class(es)
- #2020079: Multi-server subdirectory support
- #2096629: hook_provision_drupal_create_directories_alter by ref
- #2086009: Logo Cannot be changed / disabled
- #2069387: Support nonstandard ports on hostmaster-install command
- #2074681: Ports are hardcoded in hostmaster.profile
- #2103173: Provide an example of saving data to a site context
BUG FIXES
- #2110057: Key used for signing debian package has expired
- #1635628: do not talk about a makefile if installing over an existing platform
- #2103173: Provide an example of saving data to a site context
- #2109637: upgrade.sh script fails to checkout 6.x-2.x
- #2085077: Invalid argument supplied for foreach() hosting.ip.inc:39
- #2102341: Differentiate between a feature being disabled vs absent
- #2029799: Error deleting a platform: Call to a member function succeed() on a non-object
- #1971348: Site aliases get chopped if longer than 50 characters
- #2071317: Incorrect SSL IP is deployed to servers in a web pack
- #2098289: Backup task does not detect failed database dump
- #2098389: subdirs kicks in even when subdirs module is disabled.
- #1189576: can't disable the DNS service from the frontend
- #1931000: Missing drush backend output in frontend log
- #2077793: Queue daemon collecting quotes
- #2097689: clean_urls don't work well with subdirectory
- #2097363: Add site spins forever
- #1387578: hosting_queues_cron_cmd makes unwarranted assumptions about bash availablility
- #2084963: SSL vhost does not respect Alias redirects
- #2061509: List of platforms is no longer updated to match install profile clicked at node/add/site
- #2046783: Task dialogs won't open after clearing the cache
2.0-rc4 release notes
An error in the manual portion of our release process resulted in the incorrect version of the code-base being built. See the release notes for 2.0-rc3 for relevant changes since 2.0-rc2.
Known bugs
In addition to those listed in the release notes for 2.0-rc3, note that some people have experienced problems with downloading a patch in the main Aegir makefile. This is most likely due to problems with drupal.org's SSL certificate. If you are experiencing such an issue, there is a simple work-around: edit the aegir.make file in the provision/ directory to make the path use http:// instead of https://.
For a Debian-based system (including Ubuntu), this would look something like:
$ sudo vi /usr/bin/drush/commands/provision/aegir.make
For 'manual' installs on other OS's, it would probably be:
$ sudo vi /var/aegir/provision/aegir.make
Change the line:
projects[drupal][patch][] = "https://drupal.org/files/common.inc_6.28.patch"
to read:
projects[drupal][patch][] = "http://drupal.org/files/common.inc_6.28.patch"
Then re-run the install. On Debian/Ubuntu machines:
$ sudo apt-get install aegir2
On manually installed systems (CentOS, etc.) as the aegir user:
$ drush hostmaster-install
Detailed list of changes
API CHANGES
None.
NEW FEATURES
None.
BUG FIXES
- Fixed makefiles to point to proper versions of projects.
2.0-rc3 release notes
The Aegir team is pleased to announce the third release candidate for the upcoming Aegir 2.0 release.
This release mainly fixes a couple critical bugs in 2.0-rc2 that were causing problems when sites were being cloned or renamed. We've added a couple small features, such as additional options during hostmaster-install, and allowing the redirect to the Welcome page to be turned off. Also, we've made a small addition to the API, in that there is now a hook available (hook_provision_drupal_create_directories_alter()
) to allow altering the directories created when a site installed.
In order to better support PHP 5.4, we have added a patch to Drupal core to suppress E_STRICT warnings. These were harmless, but annoying. They were caused by a changed in PHP error-reporting in PHP 5.4, and the Views module's maintainers' desire to remain backward compatible with PHP 4. For more details, see: #2060727: Patch Drupal core to suppress E_STRICT warnings on PHP5.4.
A number of pending issues require a new stable release of Drush (5.10), so we will work with the maintainers to help get a new release out.
Known issues
Our release process is still done by hand instead of being more automated because we can't use our aegir distribution as core (issue #1991764).
Issue #1931000: Missing drush backend output in frontend log has been fixed, but may require a patch to Drush for the use of hosting-queued.
Subdirectory support is still very preliminary and needs a number of issues to be fixed before completion, see #2046167 for details. Note that this will not block a stable release, since this feature is 'experimental', and additional re-factoring can occur during the stable release cycle.
Detailed list of changes
API CHANGES
- #1283738 by halcyonCorsair, cweagans: Allow other commands to add or alter the directories to be created.
NEW FEATURES
- #2069387 by cweagans, mstenta: Support nonstandard ports on hostmaster-install command.
- #2067617: Allow hostmaster-install to accept '--working-copy' option.
- Allow the redirect to the welcome page to be turned off.
BUG FIXES
- Add E_STRICT patch to test openatrium makefile so tests will pass on PHP 5.4.
- #2067603: Fix original and cloned site pointing to the same database.
- #2048653: Ensure mysql is secure before proceeding with hostmaster install.
- #2060727: Patch Drupal core to suppress E_STRICT warnings on PHP5.4.
- #2038279: Warn of invalid account email on site install.
- #2055949: Fix migrate drops wrong database when domain name changes.
- #2074681 by cweagans, mstenta: Fixed ports are hardcoded in hostmaster.profile.
- Use custom functions for block visibility.
- #1940378: Fix PHP 5.4 warning by initializing an object variable prior to assigning properties.
- #2050881: Call drush.php via php, since it isn't executable when installed via PEAR.
- Enable platform site-list block, since it had to be renamed.
2.0-rc2 release notes
The Aegir team is pleased to announce the second release candidate for the upcoming Aegir 2.0 release.
This release mainly fixes our project makefile that, due to an error in our build process, did not pull in the latest code. As a result, installing 2.0-rc1 actually installed a 2.0-beta2 codebase. While changes between 2.0-rc1 and 2.0-rc2 have otherwise been minimal, we invite you to explore all the new features and bugfixes in 2.0-rc1.
Note that we have taken this opportunity to update our external dependencies, as well.
Known issues
Our release process is still done by hand instead of automated because we can't use our aegir distribution as core (issue #1991764).
Issue #1931000: Missing drush backend output in frontend log has been fixed, but may require a patch to Drush for the use of hosting-queued.
Subdirectory support is still very preliminary and needs those issues to be fixed before completion, see #2046167 for details.
Issue #2046783: Task dialogs won't open after clearing the cache was the result of updating our dependencies. Refreshing the page is usually sufficient to make modal dialogs work again. It looks like this was probably specific to the development environment, as we can no longer reproduce it.
A critical bug was discovered: Issue #2055949: Migrate drops wrong database when domain name changes. It has been fixed in 6.x-2.x, and a patch for 2.0-rc2 is available.
Detailed list of changes
API CHANGES
- Update external dependencies.
NEW FEATURES
none
BUG FIXES
- Add dependency on jquery_update, so we can use the more recent version of modal_dialog.
- Enable login and nav blocks explicitly.
- treat symlinks as existing, fixes #2046249
2.0-rc1 release notes
The Aegir team is pleased to announce the first release candidate for the upcoming Aegir 2.0 release.
This release mainly targets a few critical bugs that were found in the second beta, mostly related to ongoing Views, Drush integration and access control. We also introduce a number of new features and finally drop support for provisioning Drupal 5 sites completely (for real this time!). Along the way, we did quite a bit of API cleanup.
While we are in feature freeze, we have continued to add some UI improvements, since they don't affect the API. We've added some long-standing UI enhancements, and fixed a number of bugs in the views integration and other usability problems, as well as overhauling our access control. Significant improvements to the package-management system have also been introduced.
Our only remaining critical bugs are basically waiting on a new release of Drush, upgrade testing on Nginx and some contrib upgrade guidance documentation. If no other critical bugs are discovered during the time it takes us to fix these, we should move on to a full release of Aegir 2.0!
Known issues
Our release process is still done by hand instead of automated because we can't use our aegir distribution as core (issue #1991764).
Issue #1931000: Missing drush backend output in frontend log has been fixed, but may require a patch to Drush for the use of hosting-queued.
Subdirectory support is still very preliminary and needs those issues to be fixed before completion, see #2046167 for details.
Detailed list of changes
API CHANGES
- #1034520: Cleanup package instances when deleting sites and platforms.
- #1830220: Drop support for Drupal 5.
- #1975086: Move log parsing and status updates to seperate functions and call them from a shutdown function.
- Pass the entity type when we're sync'ing package instances.
- Save the platform field when creating a package instance record.
- Move to individual operation callbacks for VBO tasks.
- #2022849: Record disable and delete backups in the database.
- #2031491: Rename SSL permission to be more descriptive.
- Clean up hosting_node_grants().
- Add 'administer' permissions for platforms and servers, and allow platforms to be viewed.
NEW FEATURES
- #905326 by ergonlogic, crea: Improve file path changes.
- #1201174: Make UID1 username configurable.
- #1345118: Make platform access control an autocomplete form.
- #1515416 by ergonlogic, helmo, Deciphered: Replace listing of sites and platforms using a package with a view.
- #1975086: Add 'update status' button to tasks.
- #2005310: Add VBO operations to platforms view.
- #2006074 by Deciphered: Enable backups via VBO.
- #2025787 by omega8cc, ergonlogic, anarcat: Open site goto link in a new window/tab.
- #2022813 by Deciphered: Expose backups to Views.
- #2027269: Update a task's status after all Drush operations are complete.
- #2031491: Review and update permissions for all roles.
- #2031765: Clean up VBO operations with Action Permissions.
- #2035873: Show package 'popularity' on platform package view
- #2036283: Make Aegir aware of site-specific packages.
- #2036793 Ignore hidden modules and profiles.
- #2037045 by helmo, ergonlogic: Change some log statuses to better match Drush's log styling.
- #2037965: Clean up hosting-pause.
- #2038279 by ergonlogic, Jon Pugh: Validate email during site install.
- Display all validation errors when adding a client, and limit to 20 suggestions.
- Flag rollbacks as warnings in our task logs.
- Handle package page views separately for sites and platforms.
- Update default hosting_site views to use new access plugin for blocks, and add a couple more displays for non admin listings.
- Add Views access plugin for hosting_site and hosting_package.
- Add views handler to filter packages by status.
BUG FIXES
- #1238618: Fix client form validation.
- #1263264 by ergonlogic, recidive, anarcat: Specify a type when getting a package, to avoid collisions.
- #1647830 by sambonner: Fix Incorrect ownership of directories under sites/example.com/private/files/.
- #1861898 by ergonlogic, Jon Pugh: Don't hardcode the types of entities to which we can attach tasks.
- #1975086 by helmo | anarcat: Fixed updating a task when there is none.
- #2025355: view platform' permission broken
- #2026417: Disambiguate site operations.
- #2031491: Fix roles.
- #2031747: Fix Views block placement and visibility.
- #2038891: Add 'client_email' option to 'provision-install-backend'.
- #2038891: Switch from '--invoke' to '--strict=0' for backend calls.
- #2040285 debian: properly detect webserver, again
- #2044251 drush command '@none provision-save' could not be found
- #2045907: Remove extra tabs from deleted platforms.
- Fix php 5.4 issue
- Add missing unicode include for UID1 name validation.
- Fix update hook name.
- Only register shutdown function to update task status in the context of a front-end task.
- Don't display package block on profiles.
- Only show link to add clients on site form to those with proper permissions.
- Override hook_access functions for node_grants when hosting_client is enabled.
- Register node type in hosting_task's Hosting feature.
- Clean up Views exposed forms.
- Fix Views block placement and visibility.
- Only display client sites block on the 'view' tab.
2.0-beta2 release notes
The Aegir team is pleased to announce the second stabilization (beta) release of the Aegir 2.x branch.
This release mainly targets a few critical bugs that were found in the first beta, namely IP allocation on non-SSL and non-cluster sites which was completely broken and bugs the Debian package. We also introduce a few new features and finally drop support for provisionning Drupal 5 sites completely.
While we are in feature freeze, we have continued to add some UI improvements, since they don't affect the API. We've added some long-standing UI enhancements, and fixed a number of bugs in the views integration and other usability problems.
We are down to only one active critical bug. If no other critical bugs are discovered during the time it takes us to fix it, we should move on to our first release candidate!
Known issues
Our release process is still done by hand instead of automated because we can't use our aegir distribution as core (issue #1991764)
Issue #1931000: Missing drush backend output in frontend log has been fixed, but may require a patch to Drush for the use of hosting-queued.
Subdirectory support is still very preliminary and needs those issues to be fixed before completion:
- Nginx support missing (issue #2020091)
- If
example.com/foo
is created, theexample.com
virtual will be overwritten and the site inaccessible under that domain (issue #2020089) - Multi-server support is untested (issue #2020079)
- Code needs to be refactored to the new 2.x API (issue #2020075)
The move to Views has highlighted some additional issues:
Fixing #2027269: warnings and errors in front-end pre- and post- hooks are ignored for task status required a new Drush commandfile, and thus a 'drush cc drush' if upgrade between an alpha/beta release and HEAD.
The debian package still doesn't properly configure the webserver, a patch has been committed and a workaround is available in issue #2040285.
New features
- #600350: provide a nice "history" of ran tasks
- #1988642: Add site by platform
- #1912238: automatically build and upload aegir 2.x packages
- #585830: usability story: "How do I create a client?"
- #1830220: Drop support for Drupal 5 in 2.x
- #1212086: Platforms should be locked by default
- #1403208: Add additional roles
- #1174440: provide a way to show site's DB username/password in the frontend
- #606120: Disabled sites should be class hosting-warning, enabled but problematic sites should be hosting-error
Bug fixes
- #1992254: restarting the mysql server crashes hosting-queued
- #1761932: Explain that updates will require the verify task to run
- #1603702: allows creation of SSL site even if there are no IPs available
- #1130992: User 1 has lost the ability to create platforms
- #1126638: two sites can take the same server IP for SSL
- #1994000: Nginx looking in wrong place for SSL setup information
- #2023621: Unable to allocate IP address for certificate, disabling SSL.
- #1929454: hosting_cron.modules requires hosting_site.module but lacks a dependency.
- #2014589: Views-based package list omits module/theme version
- #2005246: It is possible to disable Hostmaster site via bulk site operations
- #2025533: Mod Rewrite and Apache.conf symlink not created with debian package - beta2
2.0-beta1 release notes
The Aegir team is pleased to announce the first stabilization (beta) release of the Aegir 2.x branch, after nearly 3 weeks of development since our second alpha release. While we had expected an additional alpha release prior to the beta cycle, development on outstanding features went smoother than foreseen. We've added some significant features, and fixed a couple bugs and (again) made some important improvements to our API.
This release marks our feature freeze. The intention is to limit any further changes to only critical and major bug fixes. However, exceptions may be granted on a case by case basis. Our development efforts will now move to tackling all release critical bugs. Once all of these are fixed, we'll move on to our release candidate (RC) phase. That said, we expect this release to be sufficiently stable that we intend to run it in production at Koumbit soon.
Release overview
This release introduces a major new feature into Aegir core: support for installing sites in subdirectories (example.com/foo, example.com/bar, etc.). This has been our number one feature request from higher education institutions, and so we hope that this will enable greater adoption within that sector. While categorized as an "experimental" feature, we feel our approach is sufficiently strong that we're considering merging this into our core site functionality.
Additional new features include a nice homepage, fixes to SSL support and IP allocation on clusters, and we've added a couple new roles.
We've also tackled a number of bugs, especially in the Debian package, which now supports installing Aegir on Nginx servers. We've also fixed some issues introduced with the new views code, and improved nginx support.
The project's Debian repo now includes Drush 5.9, and we have migrated the 'aegir2' packages to the Testing repository.
Known issues
- the debian package shows a lot of garbage when installing from scratch, which is almost harmless except that we don't see the login URL, use
drush @hostmaster uli
to get a new one, fixed in issue #2002076 - SSL support was severely broken for single servers and could lead to loss of the SSL certificate copies when a non-SSL site is installed. See commits f0980e0..9b86038 in Provision for fixes. (fixed in issue #2023621)
- the Debian package is severely broken - in most configuration (e.g. clean installs but not upgrade) the install will just fail with a permission denied. See commits d4efa37 and 56d1718 for fixes.
These are related to drush:
- some warnings and errors may not show up in Aegir's task logs, making it believe tasks succeeded when the actually failed. this affects only hosting-queued. (issue #1931000 / issue #1982502)
- our release process is now done by hand instead of automated because we can't use our aegir distribution as core (issue #1991764)
Subdirectory support is still very preliminary and needs those issues to be fixed before completion:
- Nginx support missing (issue #2020091)
- If
example.com/foo
is created, theexample.com
virtual will be overwritten and the site inaccessible under that domain (issue #2020089) - Multi-server support is untested (issue #2020079)
- Code needs to be refactored to the new 2.x API (issue #2020075)
API changes
- Multiple files can be managed in a single context now (see issue #2000038, issue #1784108)
New features
- Support for installing in subdirectories (issue #705026)
- Proper homepage on startup so non-logged-in users don't see an error page, and new users have basic instructions (issue #1793740)
- SSL support for clusters should now work properly (issue #2000964)
- New roles added: 'aegir platform manager', 'aegir administrator' (issue #1403208)
- Convert list of platforms to use views (Issue #1876350)
Bugfixes
- Debian package support for nginx was severely broken issue #2001142
- the views bulk operations version shipped with alpha2 had security issues, see issue #2001964
- an update hook was incorrectly named, and has since been fixed. The updates should not be destructive, but may output errors.
- Placement of a number of our new Views-based blocks was omitted from the install profile, and so won't appear on a fresh install. These can be manually placed on the blocks page.
- fix warning "Invalid argument supplied for foreach()" (issue #2005698)
- fix "Unknown options for provision-save" error (issue #1972286)
- removed hardcoded checks for IP addresses in
settings.php
that belong to core (issue #2013683) - Views filter for status does not tell what to filter for (issue #1997088)
- nginx cloaks database credentials properly now
- configure nginx to properly talk to the default php-fpm configuration in Debian (issue #1635622)
- lots of fixes for the Debian package, including
- Silence garbage in the install output (issue #2002076)
- nginx support in sudoers file (issue #2001142)
- nginx autodetection in Debian package (issue #2003712)
2.0-alpha2 release notes
The Aegir team is pleased to announce the second preview (alpha) release of the Aegir 2.x branch, after nearly 3 months of development since our first alpha release. We've squashed a number of tenacious bugs, added some nifty features, and made some important improvements to our API. We expect at least one more alpha release before we call a feature-freeze, and begin out stabilizing and cleanup (beta) release cycle.
During this release cycle, we also began work on Aegir 3, a more-or-less straight port of Aegir 2 to Drupal 7. We continue to sync changes in 2.x to our 3.x branch, and look forward to continuing the port (and beginning to add new features) as soon as we've released our first stable 2.x release.
Our next alpha release will target getting sub-site support (example.com/site1, example.com/site2, etc.) into Aegir core as an experimental feature. This has been our number one feature request from higher education institutions, and so we intend to incorporate it into Aegir 2, in order to enable greater adoption within that sector. We have a fairly detailed implementation plan, and so feel confident that we can incorporate this major feature, in fairly short order.
Release overview
This release features improvements to the IP management mechanisms that were broken in the last alpha1 release. We now manage IP addresses individually, and check if an IP is in use before allowing it to be deleted.
We also finally leave the files alone on spokes in the multi-server model, a long-standing bug in the 1.x series that led to data loss. A lot of work was done to standardize the Nginx configuration to allow for simpler, yet optional, configurations that leave the admin in charge of choosing the optimisation methods.
Also, we've moved Hosting and Eldir back to their own projects, and are now included via our makefiles. This simplifies building custom install profiles, though it complicates our release process somewhat. We've also added a drupal-org.make that will eventually allow proper distribution bundling of Aegir.
Finally, we've moved to using Views and View Bulk Operations for our lists of sites, platforms and packages. While there remain some rough edges, this will allow much easier customization of our principal UIs. It has also significantly reduced the custom code we have to maintain, in favour of exported Views, which simplifies maintenance.
Known issues
- the Debian package support for nginx is severely broken #2001142
- the views bulk operations version shipped with alpha2 has security issues, see #2001964
- the debian package shows a lot of garbage when installing from scratch, which is almost harmless except that we don't see the login URL, use
drush @hostmaster uli
to get a new one, fixed in #2002076 - an update hook was incorrectly named, and has since been fixed. This may cause the hook to be run again, if the site is upgraded to a newer version. The updates should not be destructive, but may output errors.
- Placement of a number of our new Views-based blocks was omitted from the install profile, and so won't appear on a fresh install. These can be manually placed on the blocks page.
API changes
- deprecate hosting_ip_delete_revision(), dupe of hosting_ip_delete() now that revisions are gone (8 weeks ago)
- allow running tasks that appear to be running with --force
- remove deprecated DEBUG flag in debian package
- #1785624: Some Drupal API changes in D7 (and D8) are not used/respected properly
- #1945950: Rename provision_drupal_sync_site_back()
- #1083366: Make the spokes authoritative for files/ and private/ directories
- #1812338: Refactor sync back
- a new control file has been introduced in /etc/nginx/basic_nginx.conf to force the nginx configuration to be the "simpler" one (see #1635596: nginx: do not decide the policy for users)
- #1987026: Move generated platform drushrc.php to sites/all/drush
New features
- #1929372: Flag tasks with logged warnings
- #1515416: On a package page, show table listing sites and platforms using the package
- #1853620: Add db_name to site summary
- #1681904: Ability to configure a url to redirect to in site configuration.
- #1968226: manage each IP individually on the server level - manage IPs individually
- add uninstall command
- Nginx Security: BEAST attack protection and fix for PCI compliance.
- #1980136: Allow setting default profile from the front-end.
- #588728: Replace custom lists with Views and VBO for sites, platforms and packages.
Bugfixes
- #1930670: Duplicate entry 0 for key PRIMARY in hosting_ip_addresses when installing / upgrading
- #1907028: user warning: Table 'XXX.hosting_ip_addresses' doesn't exist
- #1961920: nginx: [emerg] invalid number of arguments in "limit_conn_zone" directive in /etc/nginx/conf.d/aegir.conf
- #1923490: Incorrect error message in aegir-provision2.preinst
- #1901508: "gzip --rsyncable" is invalid on OS X
- #1678528: Database deleted on edge cases
- #1930740: provision-delete leaves a drushrc.php lying around
- Nginx: Do not override Nginx name with fake Apache name.
- #1990370: Enable Hosting feature dependencies.
Other
- #1923552: Rename 'Queue runner settings' tab to 'Queue daemon'
- #1834036: Add 'hosting platform pathauto' to the .gitignore
- #1785624: Some Drupal API changes in D7 (and D8) are not used/respected properly
- #1979496: Update upgrade.sh.txt for Drush 5
- #1974752: Document old_uri option for provision-deploy
2.0-alpha1 release notes
The Aegir team is pleased to announce the first preview release (alpha) of the Aegir 2.x branch, after almost 2 years of development. We have been pretty busy with the maintenance of the 1.x branch, so development has been sometimes intermittent, but it is not over yet!
This release is only a first of a series of alpha releases that are tailored to make upcoming changes available to a wider audience than people ready to install from git, but also to make features more widely available and tested. More alpha releases will be published on a more regular basis in the weeks to come as more features from the new provisional roadmap are implemented. We are still considering the following:
We have set the following priorities:
- subsite support (#705026) - now that mig5 did most of the work ;)
- nginx cleanup (#1635596, #1635622, #1622846, #1635586, #1608910) and Nginx Debian package (#1348560)
- files/ sync settled (#1083366)
SSL code cleanup (#941870, see also this report)- done! Now onto testing and fixing the actual bugs linked from #941870
We would like to get those in 2.x, but since no one has really started significant work on those, we are more likely to simply postpone those to 3.x at this point.
- standard archive support (#1138882, original roadmap description)
- intersite security (the infamous #762138)
- more intelligence in spokes (original roadmap description)
Also, since every single attempt at providing a proper timeline for the final 2.0 release has failed miserably, we'll just stop trying our luck and just give up on a formal timeline. Instead, we'll just do the freaking work. :) Also remember that the port to Drupal 7 has been postponed to Aegir 3. Aegir 1 and 2 both support provisionning Drupal 7 sites, but the frontend is running Drupal 6. We are aiming for a "straight port" (no rewrite) of the frontend for Aegir 3.
1. Major changes
This first alpha already packs a lot of changes. Those release notes detail the changes since the 2.x branch was first created, all the way back two years ago, when 1.0 was released. Of course, we are not detailing all the changes made in the 1.x series that are all part of the 2.x, but only the changes specific to the 2.x release.
- SSL improvements:
- IP allocation was moved to the backend to the frontend
- SSL certificate generation now has saner defaults (e.g. 2048bits)
- better error handling
Drush 5 support
- includes better support for archive-dump command
- means we also drop the dependency on drush make 2.3, now included in Drush 5
- support for Drush 4 has been dropped, the minimum version is now 5.5
Debian package improvements and changes
- Nginx support in the Debian package
- now a "native" debian package
- package name changed: it is now
aegir2
,aegir-provision2
, etc, to avoid overwriting the previous package - note that aegir 1 and 2 can not be installed in parallel
- Nginx improvements:
- fixed support for nginx 1.3 and newer
- better defaults for caching
- SSL and Nginx are now officially supported (not marked as "experimental")
- New modules:
- the
hosting-queue-runner
module is now merged into core ashosting-queued
- "pack" module, designed as a lightweight replacement to the "cluster" module, now available as an experimental extension
- the
- Hosting-queued improvements
- run with a lower priority ("niced")
- improved portability and reliability of startup script
- enabled by default in the Debian package
- Code refactoring and improvements:
- now using the Symphony autoloading code
- fix coding style in a lot of source files
- improved builtin test suite
- eldir is split in its own module again, allowing for porting it more easily to Drupal 7
A more detailed log of bugfixes and improvements is available below.
2. Major API changes
Those changes are, as usual, more explicitly documented in the upgrade path documentation.
- hosting-task now needs a
--force
argument to run a non-queued task - functions that were deprecated in 1.x are now removed
- the
email
andclient_email
database fields are now removed from client and site node types - numerous changes to the IP allocation and SSL management code
Please note that the 2.x API is still not considered stable and may change without warning until 2.x is fully stabilized, see our release process for more information.
3. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
Note that you should upgrade to the latest 1.x release (currently: 1.9) before attempting the upgrade to 2.x. This is especially important if you are running a pre-1.0 release (poor you!).
It is still imperative that you read the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade.
4. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
5. Features and improvements
In addition to the "major changes" mentioned above, the following should also be noted.
- #1830994: Login directly via 'login' link
- #1853588: Sort Install profiles on Site add form
- #709862: Allow drush modules to add to apache vhost config template
- #1861896: Make $type argument of function hosting_available_tasks() optional.
- #1843198: Add Content top and bottom to Eldir.
- #1826074: Add tests to ensure site directories are migrated
- #1464220: Support alternative 'Hostmaster' profiles
- #610040: Add X links on listing pages
- #710834: notify clients on site migrations
- #1260066: Allow use of the working copy option when creating a platform
- #1047992: exportable backups
- #1189556: Pull the queue runner into Aegir 2
- #1408410: Add microcaching to Nginx config by default
- #1138104: Tiny module to automatically fill in platform publish path based on title
6. Bugfixes
This is a non-exhaustive list.
- #1735174: server verification fails after configuration on pack slave servers
- #1798252: reload hosting-queued on upgrade
- #1898260: platform form should allow editiong if platform is not verified
- #1435098: Client name validate sometimes gives erroneous results
- #1591720: Illegal choice when selecting sites to verify
- #1587232: All radios labels in forms use incorrect vertical-align property value (middle)
- #1218130: checking "Hide platforms with non-default profiles" breaks D7's minimal and standard profiles
- #1227954: hosting_quota lacks a feature.module_name.inc
- #528064: h2 bigger than h1 in content
- #460974: improve queue admin form
- #1855486: Patch to Remove some Notices
- #1881868: SA-CONTRIB-2012-080 hasn't been applied to 6.x-2.x
- #1873384: 'Silence warning' commit broke 6.x-2.x
- #1843078: aegir 2.x debian packages fail to install
- #1829430: drush hosting-tasks broken
- #1146014: A user can see sites that don't have a client associated.
- #1256508: path aliases for deleted sites are not deleted
- #1249892: Checkboxes on provision page not aligned
- #941870: failure to copy SSL certificates should abort tasks
- #1334950: permissions on /var/aegir/config/server_NAME/SERVICE prevents access
- #1471928: provision-save: Unknown options: --context_type, --platform, --server, --db_server, --profile, --client_name, --aliases
- #1751006: Make is called incorrectly
- #1750974: Unable to pass working-copy or backend-only to hostmaster install
- #1204990: Drop old database when re-deploying a site.
- #1734500: Can't run tests with Drush 4.6
- #1433406: Make hosting (specifically hosting-dispatch) work with Drush 5
- #1690472: Declaration of Provision_Service_null::__call() should be compatible with that of Provision_Service::__call() null.php:27
- #1266484: Cannot verify Drupal 7.8 with PHP 5.2
- #1612044: upgrade from 1.7 to 2.x fails because of drush 5 command file cache
- #1358768: 6.x-2.x generates multiple Undefined property errors on hostmaster install
7. Known issues
The Debian packages were not uploaded correctly on the release day, and have been unblocked only on 2013-12-12 14:45 (UTC-5).
Older 1.x releases
This section documents all the older releases in the 1.x release cycle.
1.10 release notes
The Aegir team is proud to announce the tenth release in the stable 1.x release branch! This release ships a moderately critical security fix (see the Security fixes section below) and a number of bugfixes that have accumulated in the issue queue since the 1.9 release. Everyone is encouraged to upgrade.
We also want to announce that ergonlogic and helmo have joined the list of core maintainers supporting the core development of the Aegir project. Along with the existing commitments of the Aegir core maintainer team, Aegir is well assured of continuous development and long term maintenance.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Known issues
Upgrades may fail if the SSL certificates are actually symlinks to files not readable by Aegir, a rare but legitimate use-case. A fix is available in git, see #2046249 for more information.
Aegir 1.x depends on Drush 4.5 or 4.6, which are available only in Debian squeeze backports (old stable!), which may make fresh installs or careless upgrades impossible. To workaround this issue, install the Drush package directly from backports:
wget http://backports.debian.org/debian-backports/pool/main/d/drush/drush_4.5...
dpkg -i drush_4.5-2~bpo60+1_all.deb
See this page for other mirrors.
Because of a little fumble in the 2.0-rc2 release process documentation, a development build of Aegir 1.x has overwritten the 1.10 official release in the stable repository. Since that development build only has the fix for #2046249, we haven't restored the previous build and thefore the version number of the Debian package is 1.10+246.efcd0a3
instead of 1.10
. However, this also means that the hostmaster platform directory will be named hostmaster-1.x
instead of hostmaster-1.10
, which will make upgrades to other development releases require manual creation of the platform.
4. Features
- #1781832 by Steven Jones: Added hosting_available_tasks() should invoke alter.
- #1760962 by Jon Pugh: Added Allow any hosting task to flag itself for needing a provision-save.
5. Security fixes
- Fix access control to sites.
See also the security advisory (SA-CONTRIB-2013-059) for more information.
6. Bug fixes
- #1990370 by ergonlogic: Enable Hosting feature dependencies.
- #1435098 by Steven Jones: Fixed Client name validate sometimes gives erroneous results.
- #1585820 by Steven Jones: Fixed Can't disable the 'client" feature.
- #1573162 by mig5, Steven Jones: Fixed Drush hosting_task_()%task_rollback() invocations are never executed.
- #1513678 by Jon Pugh, fastangel: Fixed Hooks based on TASK_NAME should handle task names with dashes in them.
- #913228 by Steven Jones: Fixed Unresponsive submit button in platform migration form.
- #1441970 by Steven Jones: Fixed Client Block assigned by default to non-existing region.
- #1441970 by jlscott: Fixed Client Block assigned by default to non-existing region.
- #1597648 by jlscott: Fixed Hosting Queues Summary Block is stale.
- #2044251: Remove '@none' from call to drush_invoke_process(), since it isn't supported in Drush 4.5.
- #1678528 by helmo: Fixed Database deleted on edge cases.
- #1912666 by ergonlogic: Fixed Stable manual 6.x-1.x install test failure.
- #1912666 by ergonlogic: Fix command to run tests for 6.x-1.x.
- Nginx: Fix typo in the location regex.
- Nginx Security: BEAST attack protection and fix for PCI compliance.
- #1906900 by fall_0ut - Nginx microcaching not disabled on localized/prefixed admin URIs.
- Nginx: Make upload progress configuration compatible with latest release of Nginx extension and integration module for Drupal 7.
- Nginx: Move some dynamic directives up in the location.
- Nginx: Better protection for private URLs from bots/spiders (2)
- Nginx: Improve locations for static files.
- Nginx: Better protection for private URLs from bots/spiders.
- Nginx: Add support for http://drupal.org/project/js module.
- Nginx: Improve no-cache exceptions for known AJAX and webform requests.
- Nginx: Improve performance for dynamic requests and reduce logging 404 errors.
- Nginx: Remove duplicate location - already protected in the server template.
- Nginx: Do not use device-specific, never working paths for Boost cache.
- Nginx: Make json compatible with boost caching but dynamic for POST requests.
- Nginx: Add Wysiwyg Fields support.
- Nginx: Do not block spiders on URIs with event/calendar regex match.
- Nginx: Avoid breaking WordPress import when WP-specific URI is used.
- Nginx: Avoid caching /civicrm* and protect it from bots.
- remove perlism in sed which would never match (\s).
- explain why we do the crazy sed on dump (mysql bug)
- #1786702 by clemens.tolboom - be nice to non-aegir backups like drush archive.
- #1788398 by marvil07: Fixed Force dependency to drush < 5.
- add missing robots and files/ rules to SSL hosts.
- remove redundant paramenters to the CSR, fix email to use a standard default.
- clarify comment on self-signed certificates.
- #941870 - check for errors when copying certificates.
- improve error handling in SSL key generation.
- bump up the SSL key size to 2048 bit.
- clarify the SSL key signing code.
- don't encrypt SSL keys only to decrypt them after.
- #1741814 by wamilton: Fixed Support For Non-Alphanumeric MySQL Passwords in provision-backup, migrate, and clone.
- #1612252 by mig5, Steven Jones: Fixed 'site_offline()' variable needs to be 'maintenance_mode()' in Drupal 7.
- #1440646 by tstoeckler, cafuego, mig5: Make Drush be runnable as Root with Provision installed.
- #1734500 by Steven Jones: Fixed Can't run tests with Drush 4.6.
- move files and robots.txt rewrite rules to the vhost.
- #1108810 - protect the complete private files directory.
1.9 release notes
The Aegir team is proud to announce the ninth release in the stable 1.x release branch! This release ships two minor security fixes and various bugfixes that have accumulated in the issue queue since the 1.8 release.
We also want to announce that Omega8.cc is joining the growing list of organisations financially supporting the core development of the Aegir project. In a partnership with Koumbit.org, Omega8.cc is dedicating money to make sure Antoine Beaupré, one of the core team members, can dedicate one day a week of his time on raw Aegir development. Along with the existing commitments of Computerminds (Steven Jones), mig5 (Miguel Jacq) and the existing commitment of Koumbit.org, Aegir is well assured of continuous development and long term maintenance.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Features
No features were added in this release.
4. Security fixes
- Filter Drush log messages properly when viewing task logs.
- Properly exit after calling drupal_access_denied() on package/task nodes.
See also the security advisory (SA-CONTRIB-2012-080) for more information.
5. Bug fixes
- #1550284: Fixed Warningns after enabling 'web pack'.
- #1549210: Fixed Drush 'failed' errors are not correctly logged.
- #1503824: Fixed hosting_site_count() ignores disabled sites.
- Fix for compatibility with Nginx 1.3.0 or newer.
- #1555398: ensure the master server in a Pack is given a MySQL GRANT, if it was not also @server_master
- Add more verbosity to the notorious 'failed to fail' error, pointing to an item in the FAQ.
- #1573004: provision-migrate fails with 'Dummy connection failed to fail: mysql: option -e' requires an argument on Debian Wheezy
- #1573182: Verifying a platform should abort if it can't be bootstrapped<
- #1552228: Nginx configuration for limit_conn should be compatible with version >= 1.1.8
- #1454316: Fixed Provision recurses infinitely on reading in context.
- #1504120: Fixed Script user of remote servers is ignored
- #1205956: Document the Provision API
- Only errors 500 and 503 are supported in fastcgi_cache_use_stale (Nginx).
- #1544144: Use $server_name instead of $host in all sites/ paths to avoid broken URLs to static files for domain aliases in D7 (Nginx).
- Do not cache http auth requests. Use compact microcaching config (Nginx).
- Use separate cache for spiders/bots (Nginx).
- #1528996: Nginx microcaching should use fastcgi_cache_valid with 5s only for 403 error.
6. Known issues
No know issues at this time.
1.8 release notes
The Aegir team is proud to announce the eighth release in the stable 1.x release branch!
This release ships various bugfixes that have accumulated in the issue queue since the 1.7 release, including a lot of fixes on Nginx support.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Features
No features were added in this release.
4. Bug fixes
- #1510572: Fixed Why do we use mktime() instead of time()
- #1185690: Fixed hosting_task_log() entries are not deleted when task is.
- #1348338: Fixed Ajax requests ignore base_url() setting - followup.
- #1437856: Fixed Display of Client Quota missing pager navigation bar
- #1388542: Fixed packages in sites/all/modules should override profile/*/modules
- #1498102: Fixed Dates and times are incorrect on freshly created tasks
- #1472480: Fixed Restrict the Task Queue to one task per site each cron run
- #1493286: Fixed SSL certificates not being created
- #1485746: Fixed MySQL root password incorrectly escaped in frontend
- #1528996: Nginx microcaching should use fastcgi_cache_valid with 1s only for upstream errors
- #1472460: Nginx configuration denies HTTP request methods: PUT, DELETE and OPTIONS and it breaks services/REST
- #1534306: Duplicate directives in the Nginx config cause issues with recommended Dotdeb Nginx build
- #1528996: Nginx microcaching shouldn't cache any upstream errors
- #1524860: Empty response 444 in Nginx confuses search bots
- #1515762: Nginx microcaching should skip all known AJAX requests
- #1505370: Conflict between Mime Type and Document Type in Nginx
- #1498774: Set higher value for map_hash_bucket_size in Nginx by default
- #1283854: Fixed Use of parameters in provision_file()->symlink() is incorrect
- #1387394: Fixed deleting a site doesn't delete soft links
- #1388542: Fixed packages in sites/all/modules should override profile/*/modules
- #1486298: Fixed End of install process is messy
- #1478984: Add Access-Control-Allow-Origin header with wildcard for static files in Nginx, also to support cdn-delivered @font-face
5. Known issues
No know issues at this time.
1.7 release notes
The Aegir team is proud to announce the seventh release in the stable 1.x release branch!
This release ships various bugfixes that have accumulated in the issue queue since the 1.6 release, including a lot of fixes on Nginx support. We also ship a new clustering module aimed at lightweight slave deployments named the pack module, designed to replace the cluster module which has performance and configuration issues.
Also note that 2.x development is moving ahead with Drush 5 and Drupal 7 support, along with code refactoring and other features. Debian package builds will be available shortly in a separate Debian repository.
We also want to take this opportunity to welcome to our core team omega8cc, a long time contributor that will now be responsible for the maintenance of the Nginx code. Welcome on board Grace!
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Features
- #1463494: Radioactivity does not work with Nginx configuration
- #1353608: ad module incompatible with current nginx_simple_include.conf settings
- #1259098: nginx HTTP service should support different restart commands
- #1171508: Nginx capabilities detection is not reliable
- #1408410: Add microcaching to Nginx config by default
- #1062168: chained certificates support
4. Bug fixes
- #1439196: Don't lock the DB during backups
- #1442906: Nginx does not listen on all supplied IP addresses
- #962044: Fixing broken Boost support for the home/root location in Nginx #104004
- #1388906: It is possible to break Nginx based install with upload progress by enabling nginx_ssl feature
- #1346712: Nginx config files required as includes in vhosts are not copied when Nginx SSL feature is enabled
- #1444172: Syntax error in the help description of provision-login_reset
- #1346794: Nginx SSL declarations need to be added to redirect vhosts
- #1450134: aliases are not validated in the validate step
- #1435340: node param "created" is refreshed on node_save
- #1421356: Wrong variables for files paths hardcoded for D7 sites
- #1369608: Provision shoudl add extra config for nginx ssl hosts
- #1437386: Notices from hosting_package.module
- #1018596: Aegir forms don't work in MSIE 8.0 (?)
- #1405380: drushrc file permissions are set to 0400, perhaps should be 0440
- #1396326: Mod rewrite double escapes URLs
- #1359258: Broken DB URL when installing 1.6
- #1202960: Aegir Install Script Not Picking up db information from $_SERVER
5. Known issues
1.6 release notes
The Aegir team is happy to release the seventh stable release in the 1.x branch.
This release fixes a regression introduced in the 1.5 release that stopped users from being able to compare sites when migrating. It also contains a number of small bug fixes and documentation fixes.
We recommend you upgrade your Aegir system.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Bugfixes
- #1333978: add a conditional when validating site aliases to ensure that not only do no other sites use this alias, but that it isn't identical to the site name itself. Fixes a redirect loop
- #1330984: by Steven Jones: Fixed Long list of notices on site verify.
- #1330018: by Dane Powell: Fixed Can't compare platforms ('table doesn't exist' and other warnings).
- #1146238: by mig5: Fixed The Reset password task for Drupal 5.x sites creates not working one-time login link.
- #1322208: by helmo - fix provision-restore help information and add an example
- #1336838: Added documentation for --makefile to provision-save, and fixed the style.
4. Known issues
See also the 1.0 release notes for known issues in the 1.x series.
1.5 release notes
The Aegir team is happy to release the sixth stable release in the 1.x branch.
This release fixes not being able to install Drupal 7.8 sites on PHP 5.2, an issue causing automatic installations to fail and other minor fixes.
We recommend you upgrade your Aegir system.
1. Another special note for users of the Debian packages
We've moved the location of the Aegir packages, so you must update your apt configuration. Further details can be found the in original announcement of this change: Debian archive migrated to debian.aegirproject.org.
The Debian packages also install Drush Make using a package rather than a manual download. If upgrading, you may be prompted to remove your current installation of Drush Make to continue.
2. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
3. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
4. Bugfixes
- #1324502: by Steven Jones: Fixed Drush help broken.
- #1282280: by Steven Jones: Clean up the hosting form javascript.
- #1301440: update admin_menu to 1.8
- #1282172: by GuyPaddock: Fixed hosting_task_handler_filter_hosting_task_type.
- #1273554: Fixes Clean-up for multiple warnings.
- #1273468: Fixes Typo in variable name inside hosting_package_sync() function of hosting_package.module
- #1213442: Fixes Clients are not imported from the backend correctly.
- #1238560: Fixes Updating client throws a warning about quotas.
- #1283926: by bgm: Fixed Small improvement on the trigger/views filtering in the database dump.
- #1278720: by halcyonCorsair: Fixed missing base_url() in install_send_welcome_mail() for drupal 7.
- #1111572: by helmo, Steven Jones: Fixed Undefined index: profiles deploy.provision.inc:84.
- #1150274: by Steven Jones: Fixed Undefined index: user db.drush.inc:288.
- #1263544: by GuyPaddock, Steven Jones: Fixed 'Can only throw objects in provision.config.inc, line 75'.
- #1261006: - Follow up: Tidy docs and add missing parent method.
- #1266484: - Cannot verify Drupal 7.8 with PHP 5.2
- #983056: by Steven Jones: Fixed installer does not properly escape the mysql root password.
5. Known issues
- #1328316: failed Debian upgrades may trash the Aegir apache config - fixed in the 1.5-2 Debian package
- #1330018: Can't compare platforms ("table doesn't exist" and other warnings)
See also the 1.0 release notes for known issues in the 1.x series.
1.4 release notes
The Aegir team is happy to release the fifth stable release in the 1.x branch.
This release fixes the security vulnerability as announced by the Drupal Security Team
This release contains no other fixes or changes: this is simply a hotfix release to fix the security vulnerability.
We strongly recommend you upgrade your Aegir system.
Note that this security issue affects the Eldir theme only. The security issue does not affect your hosted sites that are managed by Aegir, and no re-verification of sites or platforms is necessary.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Bugfixes
4. Known issues
- #1266484: Cannot verify Drupal 7.8 with PHP 5.2
- #1309080: Drush make refuses to install, breaking the install - workaround:
su - aegir
drush dl drush_make-6.x-2.3 --destination=$HOME/.drush - #1128844: invalid FQDN can provoke memory exhaustion during install - workaround: make sure your FQDN has a domain part to it (ie.
hostname -f
should returnfoo.example.com
, not justfoo
)
See also the 1.0 release notes for known issues in the 1.x serie.
1.3 release notes
The Aegir team is happy to release the fourth stable release in the 1.x branch. We have found only one critical bug (Drush 4.5 support) but have also fixed so many little bugs that we figured it was about time to make a new release. Also, it's becoming a bit of a tradition at DrupalCons :)
The major changes from 1.2 are a bunch of bug fixes for clients functionality, and a fix for the wget cron method.
Thank you to those who donated to the project since the previous release: harley, lol, rink & arielqgold. We continue to accept donations in order to cover various project costs. As an open source project, we need the support of the community to ensure ongoing development and support. Koumbit is pleased to provide administrative support for this, and you can find the donation page on their site: http://membres.koumbit.org/en/civicrm/contribute/transact?reset=1&id=5, which is also linked to directly from the front page of the community site.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. New features
As per our maintenance policy, we have not added any new features in this release.
4. Bugfixes
- #1250068: Drush 4.5 breaks Aegir completely
- #1256372: Drupal 7.7 site install fails with Drush 4.5
- #1238588: Problem with client internal name prefixes
- #1237178: hosting_client_validate_suggest() appears broken
- #1236490: "View client" permission broken
- #1222208: Wget method is extremely unstable and easily locks cron task completely
- #1108810: Files in sites/example.org/private are accessible.
- #1223506: cloning a site looses client site ownership
- #1200066: hosting_get_client uses client title instead of the (unique) internal name
5. Known issues
1.2 release notes
The Aegir team is happy to release the third stable release in the 1.x branch. We have found only one critical bug (in Nginx support) but have fixed so many little bugs that we figured it was about time to make a new release.
The major changes from 1.1 are that we now support the wget cron method for D7, and have fixed a lot of issues in the Nginx support. In addition, there is now the ability to "purge" platforms that Aegir might not otherwise recognize as eligible for deletion, and have robots.txt on a per-site basis. Finally, the "wget" support for cron.php has been rewritten with drupal_http_request(), so it's much faster and will detect errors properly - no more discrepancies between Aegir's idea of when cron was ran and the site's.
We've put in place a continuous integration server (Jenkins), which has allowed us to start automated testing of the Debian packages. This helps to ensure that automated installation of the software is stable. People wishing to test the latest development version of the 1.x tree can now install packages from the "unstable" repository; more information can be found here: http://community.aegirproject.org/debian#Package_versionning
While not specific to this release, we have made extensive improvements to the documentation that can be found at http://community.aegirproject.org/handbook, as well as API documentation on http://api.aegirproject.org
Also, we've begun to accept donations in order to cover various project costs. As an open source project, we need the support of the community to ensure ongoing development and support. Koumbit is pleased to provide administrative support for this, and you can find the donation page on their site: http://membres.koumbit.org/en/civicrm/contribute/transact?reset=1&id=5, which is also linked to directly from the front page of the community site.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. New features
As per our maintenance policy, we have keep new functionalities to a minimum. We did however, give in and merged the exportable backups feature in 1.1.
- #1074932: purge option for platforms
- #1173954: Support for per-site robots.txt
- #1189398: Alter hook for hosting_get_servers
4. Bugfixes
- #1090678: wget cron method broken for d7 sites
- #1197172: Private files and X-Accel-Redirect support in Nginx is broken
- #1197182: The "alert" word used in the URL causes Nginx to send empty response
- #1182680: Backup migrate module support is broken in Nginx
- #1191934: hook_hosting_servers_titles_alter exposed some bad coding
- #1182582: When searching for node.js Nginx always displays error 404
- #1183500: The IP address in the Nginx vhost can be empty, which breaks http server configuration completely
- #982452: DB name existence checking is not correct
- #881496: Problem migrating sites with mysql views
- #908314: Deleted sites cause node alias to 404 on refresh
- #1155782: div#main in eldir moves around during page zooms
- #962188: Nginx version check works for server.tpl.php but doesn't work for/in vhost.tpl.php
- #1170498: Extra config variable is missing from nginx vhost and server templates
- #1154108: Backups from clone/migrate are not recorded in hostmaster
- #1149172: Upgrade from non-Debian install failed to backup old hostmaster platform
- #1159452: hostmaster doc groups are not generated properly
- #1145888: site's client name is reset even though I can change it
- #1143068: Allow cron.php in nginx configuration so that wget works out of the box
5. Known issues
1.1 release notes
The Aegir team is happy to release the second stable release in the 1.x branch. We have found two critical bugs during the more massive deployment of 1.0 and figured we could share the fixes with the community.
The major changes here are that 1.0 had broken the upgrade of Drupal 5 sites and was still allowing duplicate sites to be created under some circumstances.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. New features
As per our maintenance policy, we have keep new functionalities to a minimum. We did however, give in and merged the exportable backups feature in 1.1.
4. Bugfixes
1.0 release notes
The Aegir team is very pleased to announce the first official stable release of Aegir - 1.0. While it's been a mostly "open secret" that most of our releases are production ready, this one has been tested through no less than 15 alpha releases, 2 beta releases and 7 release candidates, for a grand total 24 test releases before we deliver this stable version.
Speaking of open secrets, here's a word from our founder, Adrian Rossouw:
The Aegir 1.0 release marks the end of my association with the Aegir project, as I am officially stepping down as project leader, handing the reins to Antoine, Miguel and the rest of the team.
Nearly 8 years ago, I left my job to help found Bryght, one of the first Drupal service companies, with the intention of building a hosted service with Drupal. I had a dream, and that dream was called "hostmaster". Over the next several years, the system I developed to accomplish this dream evolved into Aegir.
I started the Aegir project, in it's current form, out of a desire to ensure that my dream would be able to survive beyond my ability to dedicate myself to it. The very fact that I am writing this is evidence that it has succeeded beyond my wildest dreams.
Aegir is bigger than just one person, and I am incredibly happy and proud of the progress made, so I feel completely comfortable leaving you in the hands of my very capable co-maintainers.
Thank you to everybody for their support, and I wish nothing but the best for the project and it's future victories!
And thank you, Adrian, for this great project you have given to the Drupal community, it means a lot to us!
Now onto the release. This actual release isn't much different from the previous release candidates - you can see the list of changes from the previous RC7 release at the end of those notes. However, since this is our first major release in a long time, we figured it would be important to outline how things have changed since the old days of 0.3. For those of you who weren't there, back then installing Aegir was hard. There was no "clone", no SSL, no multiserver. You had to migrate sites one at a time and you had to create platforms yourself (no drush make!). We were still using CVS.
With 1.0, we have multiserver support, Debian packages, DNS, we're using git, drush make and we have a very dynamic community of users and shops using the project. We have ambitious goals for 2.0 and we're eager (pun intended) to start moving again! So if you were waiting for Aegir to be stable, easy to install, production-ready, now is the time. The 1.x branch will be well supported as a lot of shops are running it in production already. What are you waiting for?!
1. Major changes in this release
Since our last stable release (0.3) in august 2009, we've done an incredible amount of work. The code size nearly doubled as the development team and user base, although we don't clearly know yet how many installs there are out there.
We can already say that we've been pretty successful at fulfilling our release goals for 0.4 and the 1.0 roadmap. Even though we're missing bits and pieces, especially in DNS, and multi-server support is not as robust as we'd like it to be, there's still space for improvements and bugfixes in the 1.x branch, and we'll also start on the 2.0 roadmap once this release is out the door.
- Multi-server support
- External and multiple DB support
- External and multiple webserver support
- Webserver cluster support
- Server and services abstractions
- DNS
- Master zones created on the fly with sites
- Manual modification allowed through the provision-zone command
- Slave servers
- Support for Bind and dnsmasq
- SSL support
- multi-domain/wildcard support
- use of HTTPS can be enabled or required
- Migration improvements
- Batch migrations
- Site migration fixes your body and teaser for sites/community.aegirproject.org/ changes
- No more rebuilding the whole node_access table
- Cloning
- Site rename support
- Inter-profile migration
- Ignoring site-specific modules so that mass-migrations are easier
- Improved task management
- Limit how many tasks run at a time
- task revisions and timing
- canceling tasks
- possibility of queuing tasks from the backend
- Improved platform management
- Automated platform creation through Drush makefile or even a URL
- access control and locking
- deletion
- platforms are stored in ~/platforms
- Site management improvements
- Named backups
- Password resets
- Inter-site security (storing passwords outside of settings.php AKA cloaking)
- stopped hardcoding page cache
- bulk operations
- Views support
- local.settings.php overrides
- backup file size
- .htaccess customization
- Clean database names and users (instead of site_X)
- Other improvements
- Domain alias and redirection support
- Drush "aliases" (@site.example.com) are generated automatically
- Nginx support
- Cron through wget support
- User interface improvements
- Drupal 7 support
- Client node type simplification
- ~/clients directory
- aegir_api variable in settings.php
- Quota API
- Install and upgrade process improvements
- Aegir can upgrade and install itself, no need to go through the frontend wizards in install.php or upgrade.php
- Remote DB server support for installs and upgrades
- Debian packages -
apt-get install aegir
- The Aegir install went from being a known nightmare to a total breeze, with 46 seconds to 5 minutes install times with the debian packages and 15 minutes installs on manual installs (for unsupported platforms)
- Support for 7 operating systems and counting, we added:
- Solaris support
- CentOS support
- Ubuntu support
- Gentoo support
- FreeBSD support
- ... and that's just the confirmed installation reports, if it's UNIX, it can probably run Aegir!
- Project management
- Migrated to our own git repositories
- ... then migrated back to Drupal.org when they switched to git
- We now have our own OpenAtrium site (this site!) for the community (this means you!) to participate in
- A handbook has been created and expanded to provide solid, collaborative documentation for the project, that anybody can contribute to (it's a wiki!)
- We now have clear maintainers for different sections of the code, with 4 maintainers and counting
- We now have continuous integration with a Jenkins server testing Aegir after commits and also testing the upgrade path
- An explosion of third party modules
2. API changes, documentation, 1.x maintenance policy and 2.0
The API changes since 0.3 are too numerous to list here. However, we are now committed to maintain the whole 1.x series with a stable API. There will be no change in the existing API, although we may add in some stuff if it is really necessary. All future API changes between 1.x and 2.x will be clearly defined so that third party module can be supported.
A documentation initiative has been launched to better document the Hostmaster API. In addition, a dedicated API site (along the lines of api.drupal.org) has been put in place to ease exploring the code-base of the Aegir Project. Much work remains, so you can track progress or help out here.
Major changes to 1.x will not be committed unless they are first tested in 2.0 and merged back. We will however keep the branch opened for documentation fixes and non-invasive changes like performance enhancements and cosmetic changes to the frontend. Major problems and "aegirWTF" may also be corrected in the 1.x branch, after a shakedown in the 2.x branch.
From now on, major development will take place in the 2.x branch, which will be created shortly after this release. The roadmap for that branch is ambitious but we'll tackle things one at a time, again on feature branches for major changes so that even that branch will stay stable throughout its development cycle. We will there keep in making stable alpha releases for your testing pleasure.
3. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
4. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
5. New features
Since this is like a release candidate, we tried to limit the changes to this release to avoid breaking too many things, so not many new features. Of course, since 0.3, there are tons (see above) but this is only since rc7.
6. Bugfixes
We did fix a lot of things however, with the upgrade path and the remaining critical issues.
- #1016890: Create Server as cluster is broken after save
- #1093436: hostmaster-6.x-1.0-rc2 won't install - suspected it's due to 5.1.51-MariaDB-mariadb87-log
- #1080130: Batch migrate is broken on Ubuntu 10.04 LTS with PHP 5.3 - expected to be a reference, value given
- #993944: ssl rollback failure
- #1121406: RC6 upgrad.sh fatal issue
- #1027358: Set a variable in settings.php so that modules can check whether the site is in Aegir
- #1123224: api.aegirproject.org not updating
- #1122942: aegir.make is broken in 1.0-rc6
- #1123276: Debian pkg: /var/aegir/config is not owned by aegir
- #1115768: Aegir need help with vbox DNS settings
- #1123176: Usage of
tempfile
in upgrade.sh.txt breaks CentOS compatibility
7. Known issues
Being really open about our project, we have never hidden the fact that some things, sometimes, do not work in Aegir. Our issue trackers are public, and we've made it a point of honor not only to document clearly what is wrong in our releases as soon as we find out about it, but also to reroll new releases when we fix it.
That being said, 1.0 suffers from a number of significant issues and design flaws. This is the list of all issues marked "major" in the queue right now. Some of those issues can be fixed within the 1.x branch, some can only be fixed by refactoring, and so they will be fixed in 2.x.
- #922252: DNS has no access control - anybody can create any zone (2.x)
- #762138: Design security issue with developer access to sites' modules and themes (2.x, maybe merged in 1.x if no API change)
- #1081536: Platform verify doesn't rollback apache config file (2.x)
- #1004526: Aliases are not persisted across rename and clone (2.x, maye be merged)
- #1126638: two sites can take the same server IP for SSL (2.x)
- #1111254: The Aegir API is not fully documented (1.x)
- #1081528: moving sites/all modules breaks migration
- #955854: site can be very slow with huge number of platforms and packages (2.x, may be merged, see also #1067688: bulk operation forms are too slow with thousands of packages))
- #1001620: verifying an existing platform (and site) doesn't always work the first time (1.x, hard to reproduce)
- #908314: Deleted sites cause node alias to 404 on refresh (1.x)
- #922258: update DNS records on inter-server migration (2.x, may be merged)
- #894046: language dialog should be a drop-down, not radios (2.x?)
- #941870: failure to copy SSL certificates should abort tasks (1.x)
- #922266: rollback support in DNS (1.x)
- #1128596: version numbers in the Hostmaster issue queue are wrong (1.x, will be fixed once Drupal.org wakes up)
1.0 Release candidates
This section documents all the release candidates towards the 1.0 release.
1.0-rc7 release notes
The Aegir team is very pleased to announce the seventh release candidate of Aegir 1.0.
This release primarily fixes issues with our drush makefile which have prevented installations and upgrades of the last couple of releases. We have also fixed issues with the upgrade.sh script and the Debian packages.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from before 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or recent rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Bugfixes
4. Known issues
- upgrade.sh doesn't work in CentOS out of the box - use the manual upgrade process on that platform or apply the hotfix documented in #1123176
1.0-rc6 release notes
The Aegir team is very pleased to announce the sixth release candidate of Aegir 1.0, aka "are we there yet?"
This is yet another hotfix release to fix an issue in the upgrade.sh script and Drupal 7 support.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from before 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or rc1/2, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Known issues
- Problems remain with the upgrade path. A hotfix is available in #1121406 and the tags have been rerolled so that new upgrades do not suffer from this problem
- The makefiles released with rc4, rc5 and rc6 were broken, you need to apply this patch to the aegir.make file in the provision module, see the Applying patches article on drupal.org to apply patches. The patch needs to be applied after step 5.2 in the manual install process. The Debian packages have been fixed in
1.0~rc6-2
. See bug #1122942 for followup. - Ubuntu users are having problems installing the Debian packages - we need to have a proper backtrace to debug this issue
The Debian package are still in testing mode and feedback there is appreciated.
4. New features
We have now specific maintainers for specific project areas.
A documentation spree has been started by the newly documentation team and merged in this release, so the API documentation should be more complete. In particular, we have deprecated hosting_queues_get_arguments() and hook_provision_args() that weren't used anywhere.
- #1113508: Document hosting_queues_get_arguments and hook_provision_args
- this commit introduces an prefix for the internal name of clients, so that we can respect your existing namespace policies (default is no prefix)
5. Bugfixes
- #1118286: [DNS] Each services need specific function to create config files
- #1083710: [DNS] PHP 5.3 warning fixes
- #1119968: deleting a platform does not delete symbolic links - site and platform deletion should now be more robust
- #1119850: drupal 7 support broken
1.0-rc5 release notes
The Aegir team is very pleased to announce the fifth release candidate of Aegir 1.0!
This is a hotfix release to fix a major issue in the upgrade path. All users are strongly encouraged to skip the rc4 release and upgrade here instead.
Changes in 1.0-rc5
- fix the upgrade path (#1118558)
- fix upgrade.sh to have the right older version all the time and automatically detect drush (#1117776)
Known issues
- Drupal 7 support is broken (hotfix available in #1119850)
- upgrade.sh is broken again, use this version instead until rc6 comes out
This release is otherwise mostly identical to 1.0-rc4.
1.0-rc4 release notes
The Aegir team is very pleased to announce the fourth release candidate of Aegir 1.0!
We have again found critical issues in rc3, detailed below, and a few key features that we wanted to slip into 1.0 that were just low-hanging fruits that deserved to be released. We hope this release candidate will be all shiny and without any critical issues so we can release it as 1.0.
Update: because of a last minute API change, the upgrade path is broken and it is not recommended to upgrade to this release. A new release has been published, so use 1.0-rc5 instead. See #1118558 for more information.
Key changes in 1.0-rc4
- There is now a
~/clients
directory in which symlink are automatically created to allow easy access to all your client sites (#1115960) - A variable is now set in
settings.php
(aegir_api
) to allow contrib modules to inspect what Aegir version they are dealing with, if any. The setting is removed on export, that is, if dbcloaking is off (#1027358) - We fixed a big regression where the mass migrate form stopped working in rc3 (#1111532)
- The
private
directory of Drupal 7 is now properly protected (#1108810) - The Debian package has been tested over and over again and is now the prefered install method for people on Debian
- Consequently,
install.sh
has been removed and the manual install instructions have been revamped to be clearer, see this post for more information about the reasoning behind that change
Known issues
- DNS has no access control - anybody can create any zone (#922252)
- The site form suffers from some performance issues when you have a lot of platforms (#955854)
- Cluster servers are probably broken right now (#1016890)
- Aliases are not persisted across Migrate, (Rename), Clone (provision-deploy) (#1004526)
- Upgrade path is broken (#1118558)
See all the issues tagged as aegir-0.4 and issues marked as 'major' for our comprehensive list of outstanding issues.
Installing and upgrading
The canonical source of installation documentation has moved from the usual INSTALL.txt to the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation has now been moved to the community site at
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes. You can download the upgrade.sh.txt script from http://community.aegirproject.org/node/431.
It is still imperative that you read the the Upgrade Guide (http://community.aegirproject.org/upgrading) and follow all version-specific upgrade instructions located at the end of the document before trying to run the upgrade script. This especially applies to users upgrading from before 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or rc1/2, there are unlikely to be any version-specific manual steps required to upgrade to 1.0-rc4, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
The upgrade.sh script attempts to backup the existing backend components, download new versions, and then run the hostmaster-migrate command. It assumes you are upgrading from the previous release (hostmaster-6.x-1.0-rc3). If you are not, you may have to edit the script to change the OLD_DRUPAL_DIR variable.
Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
New features
- #1115960: Moving a site between platforms causes confusion
- #1027358: Set a variable in settings.php so that modules can check whether the site is in Aegir
- #1111932: Move the suggestion of a client title into it's own function (for use when creating clients programatically)
Bug fixes
- #1111532: Mass Migrate fails in Aegir 1.0 RC3 - regression from rc2 caused by the security fix below
- #1117296: cross site request forgery issue in tasks - was fixed again due to the above regression
- #1108810: Files in sites/example.org/private are accessible.
- #1102620: provision-deploy fails silently on unavailable database
- #1111572: Undefined index: profiles deploy.provision.inc:84
1.0-rc3 release notes
Aegir 1.0-rc3 release notes
The Aegir team is very pleased to announce the third release candidate of Aegir 1.0!
This is the third Release Candidate from which we will test fixes to critical bugs from rc2, as well as fix any additional critical bugs found since, until we have no criticals and will then roll out the official 1.0.
This release also includes a fix for a security vulnerability (see below). We recommend you upgrade your Aegir instance to apply this fix.
Key changes in 1.0-rc3
A security vulnerability was fixed that prevents arbitrary configuration from being injected into Apache vhosts by way of the Site Aliases form. See #1098304
The 'Client' feature / node type has been significantly refactored and simplified, with the ultimate aim of hooking it into other CRM/databases (e.g. for LDAP support), and better Aegir-managed site/user security. This involved an exception to our current 'API freeze' for the sake of getting this important refactoring in. See this summary of the changes this imposes
We've upgraded to use Drush 4.4 and Drush Make 2.2 on fresh installs and upgrades. These have been tested and should work without any problems.
Known Issues
- DNS has no access control - anybody can create any zone (#922252)
- The site form suffers from some performance issues when you have a lot of platforms (#955854)
- Cluster servers are probably broken right now (#1016890)
- Aliases are not persisted across Migrate, (Rename), Clone (provision-deploy) (#1004526)
- The provision.info version tag wasn't updated properly and still reads 1.0-rc2
- See all the issues tagged as aegir-1.0 and issues marked as 'major' for our comprehensive list of outstanding issues.
Installing and upgrading
The canonical source of installation documentation has moved from the usual INSTALL.txt to the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation has now been moved to the community site at
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes. You can download the upgrade.sh.txt script from http://community.aegirproject.org/node/431.
It is still imperative that you read the the Upgrade Guide (http://community.aegirproject.org/upgrading) and follow all version-specific upgrade instructions located at the end of the document before trying to run the upgrade script. This especially applies to users upgrading from before 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or rc1/2, there are unlikely to be any version-specific manual steps required to upgrade to 1.0-rc3, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
The upgrade.sh script attempts to backup the existing backend components, download new versions, and then run the hostmaster-migrate command. It assumes you are upgrading from the previous release (hostmaster-6.x-1.0-rc2). If you are not, you may have to edit the script to change the OLD_DRUPAL_DIR variable.
Bugs fixed
- #1050730 : BIND needs write access on slave zones directory
- #1102650 : Check db connection before can_create_database()
- #1100196 : API docs example have service and service type mixed up
- #1097612 : PHP warnings thrown when a server has no available services.
- #1106768 : Client name of first client (created on installation) is empty
- #1098304 : Domain aliases are not sanitized and it is possible to inject arbitrary vhost configuration via aliases textarea
- #730424 : Putting credentials in Apache VHost environment is insecure
- #1067030: Using human readable name instead of machine name
- #1066676: Picture upload doesn't work using a site alias with D7
New features
- #1096498 : Managers should have access to all sites, tasks and clients
- #962330 : refactor and clarify the fields in the client content type
- #334416 : security between sites (settings.php protection)
Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
1.0-rc2 release notes
The Aegir team is very pleased to announce the second release candidate of Aegir 1.0!
Yes, you read that right: 1.0! We've decided to rename the 0.4 branch to 1.0.
This is the second Release Candidate from which we will test fixes to critical bugs from rc1, as well as fix any additional critical bugs found since, until we have no criticals and will then roll out the official 1.0.
Key changes in 1.0-rc2
There are a lot of bug fixes and features added into this release.
INSTALL.txt and UPGRADE.txt, along with the various architecture hints, and other developer docs, have been moved to the community site under:
http://community.aegirproject.org/administrator-manual
The Aegir git repositories have moved back to drupal.org, we're back in the family!
We have fixed some key issues that were discovered during testing, including sites that were not put offline during migrations and data loss issues in multiserver migration. We have also worked on the upgrade path for legacy (pre alpha9) installs, which should be able to upgrade to this release directly.
Finally, we fixed a CSRF attack on the task form that allowed an attacker to force a user to run tasks on existing sites and platforms. This is non-critical as only tasks with no confirmation dialog (ie. safe tasks) could be triggered.
Also, we've moved to Drush Make 2.1!
Known Issues
- DNS has no access control - anybody can create any zone (#922252)
- The site form suffers from some performance issues when you have a lot of platforms (#955854)
- Cluster servers are probably broken right now (#1016890) *Aliases are not persisted across Migrate, (Rename), Clone (provision-deploy) (#1004526)
- See all the issues tagged as aegir-1.0 and issues marked as 'major' for our comprehensive list of outstanding issues.
Installing and upgrading
The canonical source of installation documentation has moved from the usual INSTALL.txt to the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation has now been moved to the community site at
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes. You can download the upgrade.sh.txt script from http://community.aegirproject.org/node/431 (note, we point to the version in the 6.x-1.x branch because we accidentally rolled the tagged release before changing the AEGIR_VERSION number in the script. So use this version of the upgrade.sh script, or pay close attention to what version the script says it will upgrade to.)
It is still imperative that you read the the Upgrade Guide (http://community.aegirproject.org/upgrading) and follow all version-specific upgrade instructions located at the end of the document before trying to run the upgrade script. This especially applies to users upgrading from before 0.4-alpha8, including 0.3.
For users coming from 0.4-beta1, beta2 or rc1, there are unlikely to be any version-specific manual steps required to upgrade to 1.0-rc2, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
The upgrade.sh script attempts to backup the existing backend components, download new versions, and then run the hostmaster-migrate command. It assumes you are upgrading from the previous release (hostmaster-0.4-rc1). If you are not, you may have to edit the script to change the OLD_DRUPAL_DIR variable.
Bugs fixed
- fix CSRF attack on task run links
- #1075108 : Fix for admin/hosting/queues UI for Safari and Chrome
- #905326 : Cloning a site doesn't handle file path change in the variables table
- #1079274 : Sites data get lost on migrate or clone / dont "spoke" the site
- #1075132 : Autocomplete field 'Associate a client to this user' in user form throws a warning
- #1075088 : Users have no access to any site if "create site" permission is turned off
- #1074414 : wrong use of drush_backend_invoke
- #949044 : It is possible to create broken site or duplicate sites nodes because there is not enough strtolower(trim(domain)) used
- #1068280 : Update order issue during migration from 0.3 to 0.4-beta2
- #1068660 : site is not put offline during migration
- #884090 : aplha11 new sites get wrong group perms in files subdirs
New features
- #1062070 : We no longer rebuild the node access table all the time, only if necessary
- #1080118 : Add a page on Aegir to the d.o Administration Guide
No Drush 4?
Some people might be wondering why we haven't changed the install.sh or upgrade.sh scripts to fetch the latest Drush 4.2 release.
Drush 4.2 has a bug which breaks rsync and possibly other stuff when using multiserver.
It's been fixed in Drush, but we are awaiting a 4.3 release before we roll with it. If you see 4.3 come out, you can probably upgrade your Drush if you wish (we tested this morning and are in fact compatible with Drush HEAD at time of writing).
See this ticket for more: http://drupal.org/node/1041386
We are working with the Drush developers to attempt to 'sync' our releases in future, but we wanted to roll out rc2 to all you good folk sooner rather than later this time :)
Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always!
0.4-rc1 release notes
The Aegir team is very pleased to announce the first release candidate of Aegir 0.4!
This is the first rc from which we will go into crazy bugfixing mode until we have no criticals and will then roll out the official 0.4.
Key changes in 0.4-rc1
There are a lot of bug fixes and features added into this release.
Significantly, a major data-loss bug was fixed in Multiserver environments, that was causing a remote site's 'files' directory contents to be deleted if those files didn't already exist on the 'master' copy of the site on the hostmaster system.
Also, we stopped hardcoding "$conf['cache'] = 1" in sites' settings.php, returning control over the caching function back to the user by way of:
- inserting the preferred setting into a file called 'local.settings.php' alongside the settings.php so it won't be clobbered by the template, or
- same as above but in /var/aegir/config/includes/global.inc to set it globally for all sites, or
- setting it in /admin/settings/performance on your site, and having that setting stick instead of being reset by the hardcode.
Known Issues
- DNS has no access control - anybody can create any zone (#922252)
- The site form suffers from some performance issues when you have a lot of platforms (#955854)
- Cluster servers are probably broken right now (#1016890)
- Aliases are not persisted across Migrate, (Rename), Clone (provision-deploy) (#1004526)
See all the issues tagged as aegir-0.4 and issues marked as 'major' for our comprehensive list of outstanding issues.
Installing and upgrading
The canonical source of installation documentation is as usual the excellent INSTALL.txt. The hostmaster and provision tarballs have the following checksums:
provision-0.4-rc1.tgz: 6f4da4ba7dcfaa1bcc3c44f96beb3eda
hostmaster-0.4-rc1.tgz: b2a42b151b817c1ea6cdf15462aa2e15
For users who have Rackspace Cloud accounts and want to install Aegir in the cloud, you are welcome to try mig5's Frigg script, which automatically creates a new virtual server and installs Aegir, with one command!
For users of 0.4-beta2 and earlier, we offer an upgrade.sh script in Provision, which tries to automate much of the steps outlined in the UPGRADE.txt.
It is still imperative that you read the UPGRADE.txt and follow all version-specific upgrade instructions located at the end of the document before trying to run the upgrade script. This especially applies to users upgrading from before 0.4-alpha8, including 0.3.
For users coming from 0.4-beta1 or beta2, there are unlikely to be any version-specific manual steps required to upgrade to 0.4-rc1, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
The upgrade.sh script attempts to backup the existing backend components, download new versions, and then run the hostmaster-migrate command. It assumes you are upgrading from the previous release (hostmaster-0.4-beta2). If you are not, you may have to edit the script to change the OLD_DRUPAL_DIR variable.
Bugs fixed
- #1057736 : Cloning SSL sites will lead to a non SSL site .. but certificate is still copied
- #1057178 : Call-time pass-by-reference has been deprecated task.hosting.inc
- #1059328 : Restore and Backup_Delete tasks submit dialog even when no backups are available
- #1057280 : Site batch api error
- #1058650 : Web_group 'www' missing in $common_groups array
- #1047922 : Drush 4.x and 5.0-dev/HEAD breaks Aegir in some (hidden) ways - paths in files not rewritten on clone/migrate/rename
- #931420 : Task list in hosting_summary block shows underscores
- #1052028 : [DNS] Creating and deleting zone configuration on slaves doesn't work with PHP 5.3
- #1053834 : [DNS] error adding a new record on existing zone
- #1033072 : migrate interface takes forever (litterally) to load
- #976300 : web server migration results in content removal when site is verified
- #888194 : Trying to get property of non-object provision.inc:57
- #775850 : Openpublish 2.0 Installation profile is not seen by Aegir
- #996578 : Install Profiles disappear from profile list on load or refresh of site creation form
- #1003666 : If site quota is at 100% editing site nodes is not possible
- #1039010 : Enforce that platform names and paths must be unique, even across remote servers, because of the spoke model
- #959852 : Add new server confirmation text indicates wrong server type
- #1037222 : upgrade.sh and upgrade.txt ambiguous about drush and drush make versions
- #985518 : Default client changes if Clients feature was ever enabled (0.4-beta1)
- #1014638 : All Drush versions since 4.0-rc6 are breaking Aegir completely
- #1009386 : /var/aegir/platforms logic syncs all platforms to the remote web server
- #1008954 : Fatal error: Call to undefined function hosting_site_list
New features
- #994210 : Add support for some common files/subdirs like css/js/ctools/imagecache
- #984396 : we stopped hardcoding page cache in settings.php
- #630336 : billing support
- #1005014 : Bulk operations - doesn't display all available tasks
- #1044938 : Allow local.settings.php to override global.inc to make it usable
- #1042758 : Display filesize on Backups lists
- #860582 : Get site by alias as well as url
- #1039168 : after creating platform, publish path should not be changeable
No Drush 4 or Drush Make 2.0?
Some people might be wondering why we haven't changed the install.sh or upgrade.sh scripts to fetch the latest Drush 4.2 release or Drush Make 2.0.
Drush
Drush 4.2 has a bug which breaks rsync and possibly other stuff when using multiserver.
It's been fixed in Drush, but we are awaiting a 4.3 release before we roll with it. If you see 4.3 come out, you can probably upgrade your Drush if you wish (we tested this morning and are in fact compatible with Drush HEAD at time of writing).
See this ticket for more: http://drupal.org/node/1041386
Drush Make
Drush Make 2.0 has a bug that prevents building platforms from distributions that didn't come from drupal.org.
This has also been fixed in HEAD, but we are awaiting a 2.1 release before we roll with it. If you see 2.1 come out, you can probably upgrade your Drush Make if you wish.
See this ticket for more: http://drupal.org/node/1059238
We are working with the developers of these projects to attempt to 'sync' our releases in future, but we wanted to roll out rc1 to all you good folk sooner rather than later this time :)
Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always!
1.11 release notes
The Aegir team is proud to announce the eleventh release in the stable 1.x release branch! This release ships a moderately critical security fix and one small upgrade bug with secret SSL certificates. Everyone is encouraged to upgrade.
1. Installing and upgrading
The canonical source of installation documentation is on the community site at:
http://community.aegirproject.org/installing
In a similar fashion, the upgrade documentation is:
http://community.aegirproject.org/upgrading
Within those sections you'll find step-by-step instructions for performing both manual and automatic upgrade processes.
It is still imperative that you read the the upgrade path and version-specific information and follow all version-specific upgrade instructions before trying to run the upgrade script or manual upgrade. This especially applies to users upgrading from releases prior to 0.4-alpha8, including 0.3.
For users coming from the 0.4 betas or rc releases, there are unlikely to be any version-specific manual steps required to upgrade, but you should make a habit of reading them anyway just to make sure. No-one likes a nasty surprise!
2. Need help?
If you struggle to install or upgrade your Aegir system, you have a number of options available to you for getting help.
Consult this page for more information: http://community.aegirproject.org/help
Thanks to our awesome community for their help, support and encouragement as always! Enjoy the new release :)
3. Known issues
Aegir 1.x depends on Drush 4.5 or 4.6, which are available only in Debian squeeze backports (old stable!), which may make fresh installs or careless upgrades impossible. To workaround this issue, install the Drush package directly from backports:
wget http://backports.debian.org/debian-backports/pool/main/d/drush/drush_4.5...
dpkg -i drush_4.5-2~bpo60+1_all.deb
See this page for other mirrors.
4. Features
No new feature.
5. Security fixes
- fix mild security issue with login-reset links (#1220062)
- security fix: SA-CORE-2013-003, files/ protection (#2141799)
6. Bug fixes
- fix upgrades if the SSL certificates are symlinks to files not readable by Aegir, a rare but legitimate use-case, see #2046249 for more information.
0.4-beta2 release notes
The Aegir team is proud to announce the second beta release of Aegir 0.4.
This release primarily is a bugfix release.
There have also been quite a few patches sitting in our issue queues for review for a very long time, and we have attempted to commit a whole bunch of those in this release. Things have been moving a bit slowly and we're sorry for that.
Key changes in 0.4-beta2
There are a few changes in this release that are new, such as the presence of an early 'bulk operations' feature. We do not expect this to be bug free, but hope that by committing some of this new code and making it present in a new release, we'll get some community users testing this stuff and reporting back / patching it to make it better. Otherwise this stuff just gets lost in the queue!
After this release we will be looking strongly at a release candidate for 0.4.
Known Issues:
Some of our issues from beta1 are still actually present:
- DNS has no access control - anybody can create any zone (#922252)
- Inter-server migration may fail under some circumstances (#976300)
- The site form suffers from some performance issues when you have a lot of platforms (#955854)
Also, an issue has been identified with remote web servers: in some cases, when the 'master' server syncs the site contents to a remote, it can delete the files/ of a site because it doesn't sync a site's files back to the master server except in the case of backups/migrations.
We would like to warn users that are making use of Aegir's multiserver features to be aware of this issue: http://drupal.org/node/976300 .
The road to 0.4 and Aegir contributors
Our core dev team is small and various demands for our time have been slowing development to a degree. We have a decent-sized community with a wide range of skills, and would like to take this opportunity to encourage our users to attempt to contribute to the project where they see fit - whether this be code development (bug fixes, new features), documentation, or even promotion / talks at Drupal events.
We will potentially be seeking new core developers to the project soon as well. If you'd like to chat with us about contributing to Aegir, don't hesitate to write on the mailing lists or come and see us on IRC in #aegir at irc.freenode.net. Aegir's core developers are Adrian Rossouw (Vertice), Antoine Beaupré (anarcat) and Miguel Jacq (mig5).
We know the barrier to entry can be a little high with hacking on Aegir, and will be refocusing our efforts to try and make that easier for new starters. An API site will be along soon!
See all the issues tagged as aegir-0.4 and issues marked as 'major' for our current outstanding issues.
Installing and upgrading:
The canonical source of installation documentation is as usual the excellent INSTALL.txt.
For users who have Rackspace Cloud accounts and want to install Aegir in the cloud, you are welcome to try mig5's Frigg script, which automatically creates a new virtual server and installs Aegir, with one command!
For users of 0.4-beta1 and earlier, we offer an upgrade.sh script in Provision, which tries to automate much of the steps outlined in the UPGRADE.txt.
It is still imperative that you read the UPGRADE.txt and follow all version-specific upgrade instructions located at the end of the document before trying to run the upgrade script. This especially applies to users upgrading from before 0.4-alpha8, including 0.3.
The upgrade.sh script attempts to backup the existing backend components, download new versions, and then run the hostmaster-migrate command. It assumes you are upgrading from the previous release (hostmaster-0.4-beta1). If you are not, you may have to edit the script to change the OLD_DRUPAL_DIR variable.
Bugs fixed
- #695244 : hosting-cron killed my server
- #983056 : install.sh does not properly escape the mysql root password
- #967528 : Custom modules can't specify own tasks
- #791262 : Color module & theme logo/favicon paths not updated during site deployment
- #1000890 : default_restart_cmd() doesn't return the correct restart command
- #767008 : New users aren't associated properly with their respective client nodes on site signup
- #908202 : Author not set on site node creation (and other hosting content types)
- #994212 : README.txt contains out of date link
- #988670 : Installation via drush fails on MariaDB
New features
0.4-beta1 release notes
After more than a year of (15!) alpha releases, we're finally out of the darkness and into the light! The Aegir project is pleased to announce the first beta of the 0.4 series. This release is primarily a bug fix release, but also new features that were the last targets for the 0.4 release goals.
The beta marks the beginning of the final cycle in the release of the 0.4 branch. We consider this release to be mostly feature complete. It still has bugs and problems, but since we are using it in production, we feel it's more appropriate to call it a beta, to invite wider testing and stabilise the codebase.
This is also an opportunity to test the upgrade path from pre alpha9 releases, which seem to be suffering from bitrot. You should be able to upgrade to this release from 0.3 and above.
Key changes in 0.4-beta1
- Installer now support remote db servers (#973910)
- Some work was done to improve the usability of the hostmaster-install command, which should be usable out of the box once provision and drush are installed
- By default, we setup local MySQL server to use sockets, which creates a separate server but improves security significantly, as we do not force people to open their database server to the network (#977024)
- The upgrade path for pre-alpha9 releases has been fixed
Known Issues:
The following are known critical issues with the current release, for which there are tickets, but we didn't feel were critical enough to prevent the beta release.
Some of the issues have been present since prior to alpha15, they aren't all new issues.
- DNS has no access control - anybody can create any zone (#922252)
- Inter-server migration may fail under some circumstances (#976300)
- The site form suffers from some performance issues when you have a lot of platforms (#955854))
See also the complete list of critical issues.
The road to 0.4
We are not yet in a code freeze, as we still have critical issues opened (see below) and are aiming at a few more features to be really complete (see all the issues tagged as aegir-0.4 and issues marked as 'major').
We also have still a lot of issues to review before our final 0.4 release. If people are interested in contributing, a good and painless way of doing so is to take issues in the needs review queue and test the patches. Don't forget to change the status accordingly!
Installing and upgrading:
The canonical source of installation documentation is as usual the excellent INSTALL.txt.
For the first time, we offer an upgrade.sh script in Provision, which tries to automate much of the steps outlined in the UPGRADE.txt.
It is still imperative that you read the UPGRADE.txt and follow all version-specific upgrade instructions located at the end of the document before trying to run the upgrade script. This especially applies to users upgrading from before 0.4-alpha8, including 0.3.
The upgrade.sh script attempts to backup the existing backend components, download new versions, and then run the hostmaster-migrate command. It assumes you are upgrading from the previous release (hostmaster-0.4-alpha15). If you are not, you may have to edit the script to change the OLD_DRUPAL_DIR variable.
The script is in its infancy and has not been tested except by the developer writing it. We urge caution (make backups!) but encourage feedback. If in doubt, just follow the UPGRADE.txt steps and perform all the manual commands as you usually would. Some users report problems using the script on servers with really old versions of drush_make: the script stalls after upgrading Provision. You can try to run the upgrade process manually to fix that issue, which should now be fixed in beta1.
Bugs fixed
- #978784: Using --aegir_db_host=localhost results with broken Aegir install
- #968270: @hostmaster provision-install freeze in grant_host function if database access is open
- #969326: Schema prevents install on MariaDB
- #973826: upgrade path woes from alpha6
- #898452: INSTALL.TXT and UPGRADE.TXT should mention umask
- #905386: 0.4-alpha12: provision-deploy fails with: Failed to extract the contents of /var/aegir/backups/xxxx.tar.gz to (The target
- #826840: Save settings.php with uncloaked credentials on backup (for using on non-aegir hosts)
- #966888: upgrade.sh.txt freezes for old drush make (from 0.4-alpha7)
- #965256: INSTALL.txt doesn't leave MySQL configured correctly
- #894024: we rely on a DNS misconfiguration
- fix upgrade path from pre-alpha8 releases (hopefully)
- fix the client ACL form in the user profile
New features
- #952144: support local socket connexions for mysql
- #973910: remote db servers installs and upgrades support
- allow clients to cancel their own tasks
- #678636: ability to delete tasks from the frontend
0.4-alpha15 release notes
Aegir 0.4 Alpha15 released
The Aegir project is pleased to announce our 0.4 alpha 15 release. This release is primarily a bug fix release, which aims to tie off a huge amount of outstanding bugs and other tasks.
We did aim to release beta1 but decided that there were a few critical bugs still lingering that we wouldn't want to overshadow in such a momentous release. Instead we've chosen to push out one last alpha before (finally) heading out into beta and RC stages of the 0.4 cycle.
community.aegirproject.org
In our biggest news, which we've chosen to announce in this release, we are very proud and excited to announce the launch of our new community site, http://community.aegirproject.org !
Much preparation and planning has gone into building a clean, feature-rich community portal for our ever-growing Aegir users. We hope you'll agree that the documentation space, development information and community resources are long-awaited and desperately needed features to help users understand how Aegir works and how to use it.
The community portal features:
- News area (blog, announcements)
- Almost all the Aegir documentation has been re-written and dramatically improved in a new Aegir handbook, covering all aspects of installing, using, understanding and developing Aegir.
- Screencasts and videos from sessions and tutorials (all the historical ones have been collated and added)
- Events calendar
- Shop resources (to be completed)
- Feed syndication of the Aegir universe (twitter, articles, git commits)
The Documentation on groups.drupal.org is largely out of date and not in any sane order, and will be decommissioned in favour of the new documentation. Some of the still relevant and well-written documentation was dragged across and cleaned up where necessary.
The documentation area on the new site is still a wiki so we would like to encourage users to continue to keep them up to date and add more to help improve the experience for other users.
We'd like to thank all the contributors to the existing docs for their hard work and initiatives and hope you'll enjoy the new space and structure to keep helping make Aegir awesome.
Key changes in 0.4-alpha15
In our 0.4 alpha 15 release, these notable changes/improvements have occured:
- The DNS feature has been implemented and is available in /admin/hosting/features (Experimental)
- install.sh script has been changed considerably - most of the MySQL tests and user creation have been moved into provision to be handled by drush/php. There is no longer an 'aegir_root' user required for MySQL, we simply use your MySQL root user, which must have a password. You are required to supply a valid FQDN on installation, so that remote web servers can theoretically resolve the Aegir server if connecting to it for databases.
- upgrade.sh script added (see below, adventurous people please test)
- UPGRADE.txt updated to be a provide a little more clarity
- There's a new /admin/hosting/settings area with misc settings/customisations, including:
- ability to run wget instead of drush cron
- delete sites without having to disable them first
- when selecting a core profile, hide platforms that incidentally have those core profiles but are intended as distributions (i.e OpenAtrium)
- backup task saves site's settings.php with uncloaked creds. Useful for exporting to non-aegir servers or for importing into new aegir servers
Installing and upgrading:
The canonical source of installation documentation is as usual the excellent INSTALL.txt.
For the first time, we offer an upgrade.sh script in Provision, which tries to automate much of the steps outlined in the UPGRADE.txt.
It is still imperative that you read the UPGRADE.txt and follow all version-specific upgrade instructions located at the end of the document before trying to run the upgrade script. This especially applies to users upgrading from before 0.4-alpha8, including 0.3.
The upgrade.sh script attempts to backup the existing backend components, download new versions, and then run the hostmaster-migrate command. It assumes you are upgrading from the previous release (hostmaster-0.4-alpha14). If you are not, you may have to edit the script to change the OLD_DRUPAL_DIR variable.
The script is in its infancy and has not been tested except by the developers while writing it. We urge caution (take backups!) but encourage feedback. If in doubt, just follow the UPGRADE.txt steps and perform all the manual commands as you usually would. Edit: some users report problems using the script on servers with really old versions of drush_make: the script stalls after upgrading Provision. Either run the upgrade process manually or apply this fix to your upgrade.sh.txt
Known Issues:
The following are known issues with the current release, for which there are tickets, but we didn't feel were critical enough to prevent another alpha release. Some of the issues have been present since prior to alpha14, they aren't all new issues.
- DNS - currently there is no access control
- Hostmaster install doesn't yet support remote db servers
- 'Save' button on forms under some versions of Firefox/Iceweasel on Linux is greyed out
Bugs fixed:
- #868896 : INSTALL.TXT needs updates
- #880712 : install.sh.txt not prompting for the root MySQL password on Fedora 13
- #882970 : settings.php should not be updated before a vhost file is created and verified
- #888210 : Undefined variable: missing_requirement install_6.inc:214
- #894004 : install.sh lies about mysql listen AKA move the mysql creds handling to hostmaster-install
- #896492 : Anonymous users can't view the 'hosting/disabled/$site' disabled page
- #897326 : Typo in provision.config.inc
- #898732 : Deleted sites are counted against quotas
- #911392 : Cloned site shows wrong database in UI
- #913486 : Database server service stores port '0' in the hosting_service table
- #914040 : Server crashed because hosting_site_alias table was huge
- #919848 : verifying a server queues platform verification before the server verification
- #922716 : Expand link opens new window when used from modal frame.
- #928570 : Modal Frame font is inconsistent
- #931136 : You can't delete the D7 alpha7 and beta1 sites.
- #934830 : no width limit of 320px for div#navigation ul.links
- #937490 : Fix Platform access control to deal with profile > platform paradigm
- #940816 : SSL Enabled Settings do not trigger insertion of Rewrite rules in vhost file unless domain aliases specified
- #942144 : Warnings when deleting a site
- #944450 : Saving a Package Node removes attached data
- #945894 : Enabling DNS service on server node gives MySQL errors
- #946422 : cannot sync platform to remote
- #947898 : Code in global.inc not picked up by settings.php
- #949044 : It is possible to create broken site or duplicate sites nodes because there is not enough strtolower(trim(domain)) used
- #950976 : Update hints for OS X
- #950980 : New OpenSSL breaks install script
- #951296 : @localhost no longer allowed in email addresses in Drupal 7
- #955184 : Backup task breaks any site by replacing settings.php with version working only with Apache
- #958094 : Disabling a site through Aegir front-end doesn't make it inaccessible
- #960106 : That the 'Publish path' for a new platform should be absolute is not obvious (usability)
various others not reported in the queue
Feature requests fixed:
- #366420 : DNS slave server support
- #366814 : manage DNS records properly when creating/restoring/verifying/disabling/deleting sites
- #826840 : Save settings.php with uncloaked credentials on backup (for using on non-aegir hosts)
- #896914 : Migrate form displays current platform as a target
- #897982 : Limit choice of install profiles to those on available platforms
- #903884 : Delete task should be visible and inactive, not hidden.
- #922278 : configure the master server to allow slaves to do zone transfers
- #955550 : Improve Documentation to Reduce IRC Support Requests
various others not reported in the queue.
This was a mammoth and long-awaited release. Thanks to all our users for their continuing support, patience and contributions!
-- The Aegir core devs
Previous release notes
Release notes prior to Aegir 0.4-alpha15 were published on groups.drupal.org and can be read there.
Developer Manual
This section of the documentation is aimed at developers interested in the Aegir project.
We use Git as a version control system for managing the project code, so we have specific instructions on how to use git.
Developers wishing to collaborate directly with the core team will want to read the release engineering guide.
Developers working on third-party modules should read the guide to extend Aegir and the API documentation.
Developers interested in developing on, for or with Aegir might be interested in Aegir-up, which provides a fully installed and configured virtual machine on Virtualbox/Vagrant. The Puppet module on which it is based supports an $aegir_dev_build
option, that will install Provision and Hostmaster from the git repos, preserving the .git repos.
Git repositories locations and instructions
The source code of Aegir is located on Drupal.org, as with the issue queues and release nodes.
Aegir is also made up of several separate components designed to work together, that are grouped in two main git repositories: hostmaster and provision. This page details those locations.
Newcomers to git should probably consult our git crash course while people already familiar with git can have a reminder on how to use it within aegir through the basic git workflow page.
Repositories locations
There are the URLs you can use to pull the repositories:
git clone http://git.drupal.org/project/provision.git git clone http://git.drupal.org/project/hostmaster.git
Developers with push access to the repositories should pull via SSH.
git clone ssh://[username]@git.drupal.org/project/provision.git git clone ssh://[username]@git.drupal.org/project/hostmaster.git
There is a web interface for your perusal at:
The git repositories were hosted for a while on Koumbit's servers, but were migrated back to Drupal.org during the great git.drupal.org migration. Do not use git.aegirproject.org as it doesn't have the most recent commits and may be shutdown at any time.
A crash course in Git
Clone a repo
Let's take hostmaster:
git clone http://git.drupalcode.org/project/hostmaster.git
If successful, you will have a hostmaster directory at the last state of development ('HEAD' in CVS speak, usually 'master' branch in git speak, though in our case this is now 6.x-1.x).
Check the status of the local git repository:
git status
See the commit messages:
git log
Check all 'releases'
git tag
Create a new branch for your local development:
git checkout -b <new_branch_name>
Create a new branch for development, based on an existing remote branch or tag
git checkout -b <new_branch_name> origin/<branch_name>
Basic git workflow: committing and pushing
So say you have some work you want to do on the hostmaster module. You first need to clone the repository:
git clone http://git.drupal.org/project/hostmaster.git
That will create a hostmaster
directory with the whole history of the project, and a checkout of the "master" branch (which is the equivalent of CVS's "HEAD" in git).
You make some modifications to the code to fix a bug, and then, you make a commit. That commit will only be local for now, that's the way git works:
vi hostmaster.profile
git commit -m "#12345 fix the install profile to do X" hostmaster.profile
Note that contrarily to subversion or CVS, you need to explicitly mention which file(s) you want to commit. You can also use the -a flag to commit all changes.
If you have write access to the repository, you can now push your changes straight into git. However, the git:// url above is readonly, so you will want to use SSH to push your changes. For that, we will have had to add your SSH public key to our repositories. You may also want to add a special "remote" repository to push your changes. You will need to do this only once (per repository):
git remote add drupalorg ssh://[username]@git.drupal.org/project/hostmaster.git
You can now push your changes to that repository.
git push drupalorg
If you do not have write access to the repository, you can send the Aegir developers patches by submitting them for review in the Issue queues (see the patch contributor guide on drupal.org for more details) or by uploading them to your personnal sandbox.
Branch and tagging conventions
Aegir development makes a heavy use of branches to keep development stable. We use feature branches to do large development work and have release branches to keep stable releases maintained while development happen on the development branches.
1. Summary
- The current development branch is: 7.x-3.x
- The current stable branch is: 6.x-2.x
2. Branch naming conventions
All branches are either development branches, stable branches or feature branches. Development and stable branches have the same naming convention: 6.x-2.x
or 7.x-3.x
. The only different between the two is that a stable branch is the branch where stable releases are published from.
Note: since the great git.drupal.org migration, we are back on Drupal.org and have been forced to change some of our naming convention, especially to include the core release number in the tag (so 6.x-2.x
), which is a source of confusion for our provision
module which supports any Drupal core version. See this issue on drupal.org for followup.
2.1. Main development branch
The main development branch is created after a fully stable release is published. For example, the 6.x-2.x
branch has been created when the 1.0 release was published. It is a development branch until it is considered feature-complete and goes through the alpha/beta/rc release cycle, at which point it will be declared a stable branch and a new development branch will be created (7.x-3.x
at that point).
The development branch is not stable and should not be used in production environments.
Note that we have dropped the master branch as it is not well supported on Drupal.org for releases.
2.2. Stable release branches
Stable releases are performed from stable release branches. A branch is declared stable only after it went through the complete alpha/beta/rc release cycle, as detailed above..
Once the branch is declared stable, new releases are quickly done as "hotfixes", without intervening alpha/beta/rc releases. So for example, 6.x-1.0
will come after 6.x-1.0-alpha1
, 6.x-1.0-beta1
, 6.x-1.0-rc1
at least, and then be followed directly by 6.x-1.1
.
Releases on the same stable branch are API-compatible with each other (so 1.1
is compatible with 1.0
). The upgrade path is supported between major stable releases (so you can upgrade from 1.0
to 1.1
or 2.0
).
Special case: during the 0.x release cycle, releases were not API-compatible with each other, ie. 0.4 was not compatible with 0.3, and you could barely upgrade from 0.3 to 0.4.
Only one stable branch is maintained at a time.
2.3. Feature branches
More complex development goes into development or "feature" branches, to avoid making master
too unstable. Feature branches must be prefixed with "dev-" and use the issue number (if available or relevant) and a short name. So for example, there could be work on a dev-12345-mybug
branch or dev-dns
branch for a larger feature (DNS, in that case) which doesn't fit in a single issue.
Feature branches should be based of a known stable baseline as much as possible. For example, you should branch off the last alpha release if you want to make merging with other dev branches and master easier. See this article for a thorough discussion on this practice
3. Tag naming convention
Every release is tagged with a 6.x-x.y
tag (e.g. 6.x-0.4
, 6.x-0.4-alpha15
or 6.x-1.0
). Those tags are usually laid on the stable branches, except for alpha/beta/rc releases which are laid on the development branch.
4. CVS and git history
The CVS tags and branches are still accessible in git and will be kept there for posterity. They respect the traditional Drupal.org naming convention.
They have also been converted to the new git.drupal.org conventions during the great git.drupal.org migration.
Since that migration (february 2011), history has been rewritten to clean up the authors in the history.
The great git.drupal.org migration
After thorough discussions where we weighted the pros and cons of profiting from the Great Git Migration, I think it's pretty clear that we have a great opportunity to migrate our source repositories back to Drupal.org. We will get:
- more visibility - people will find us through drupal.org easily
- more consistency - versions in the issue queue matching the real releases
- even more consistency - people will find the source better from drupal.org
- and then more consistency - usage stats will work again!
We will be able to:
- host our module and theme within the install profile - the major reason why we migrated away back then
- use whatever naming convention we want for tags and branches - although only some will be used for releases for now, see this issue for follow-up
- keep our history - the git team was nice enough to clone directly from our repositories (although see below, we're taking the opportunity of the migration to clean up our history)
What it means for you
This does have an impact on your existing repositories. Since we migrated to git.drupal.org, the repositories have changed location. The other thing is that the commit IDs have changed, and you will need to clone new checkouts before you can work with them effectively.
So in short:
- repos will move to git.drupal.org
- you need to clone again
- make sure your identity is set correctly
URL changes
So the repositories have moved. Everyone has to fetch the source code from git.drupal.org now. At some point in the future, Koumbit will close git.aegirproject.org. Concretely, Koumbit will keep the mirror running readonly for a while just to be on the safe side, but you shouldn't expect git.aegirproject.org to work and should switch URLs, as detailed below.
This documentation is taken from the Drupal.org git workflows documentation.
Readonly repositories URL change
The readonly access will change from:
git://git.aegirproject.org/provision.git
git://git.aegirproject.org/hostmaster.git
to:
http://git.drupal.org/project/provision.git
http://git.drupal.org/project/hostmaster.git
See also the git patch maintainer guide.
Core committers repositories URL change
The read/write access will change from:
ssh://gitosis@git.aegirproject.org/provision.git
ssh://gitosis@git.aegirproject.org/hostmaster.git
to:
ssh://[username]@git.drupal.org/project/provision.git
ssh://[username]@git.drupal.org/project/hostmaster.git
... notice how you will need to set your Drupal.org username in the URL. Drupal.org has good documentation on how authentication works on git.drupal.org. See also the project maintainer git guide.
Sandboxes
The migration to Drupal.org also introduces sandboxes, something we didn't have before, and which allows you to host your own branches of our modules (even if you're not a core committer) or contrib/test modules, directly on Drupal.org. See the sandbox collaboration guide for more information on that.
What if I pull/push from/to the wrong one?
It could happen before/during/after the migration that you pull or push from or to the wrong repository. git can rebase with the new repository, but unfortunately, the old commits will be mingled with the new ones.
To be able to push to the new repositories, you will need to follow instructions below to cleanup your repositories.
Rewriting history
I have taken the liberty of rewriting our collective history. There were a lot of commits with inconsistent authors: some had invalid emails, some had no real names, some had multiple different email addresses for the same person. I narrowed the list down, through extensive detective work, to a complete list of 14 distinct authors.
I have rewritten all commit authors that didn't seem right to follow a consistent pattern:
Firstname Lastname <email>
The firstname/lastname is the combination you have used in certain commits. It's not the most popular way (ie. most commits had the nickname instead of first/last), but it seems the most logical one.
The email was your most often used email or the one you have registered as such at drupal.org, if you didn't have an email address specified (in case you committed your code back in the days of CVS). (There are 6 commits from someone with a box at ovh.net that I could track, and that I can only guess are Miguel's, but I'm not sure so I have let those one fly. :)
You should make sure you have your identity set straight in your git configuration, especially your email address, if you want your contributions to be properly credited in the future on Drupal.org. In short, this means:
git config --global user.name "User Name"
git config --global user.email user@example.com
Again, see the manual for more information, especially how this maps to your Drupal.org account.
A fresh clone is necessary!!!!
Note that, because we rewrote history, you will need to clone your repository from new remote. So yes, contrarily to what was documented here before, you will need to clone from scratch.
We understand this may be annoying if you work on a bunch of branches or have local branches you want to keep. Unfortunately, our original tests didn't cover all use cases and were misleading in thinking we could just rebase existing repositories.
Other documentation
People unfamiliar with git or specifically git on Drupal.org should read the growing git handbook on Drupal.org.
People that are familiar with git should contribute to that manual. :)
Under the hood
The history rewrite was performed through git-fast-export and git-fast-import magic, with a fairly simple perl script of my own. I have cloned the current repo, filtered it and published a new version for git.drupal.org to pull. You can see the results here:
http://git.aegirproject.org/?p=export/provision.git
http://git.aegirproject.org/?p=export/hostmaster.git
The exact calling sequence is this:
mkdir export orig cd export git init --bare hostmaster.git cd ../orig git clone --bare git://git.aegirproject.org/hostmaster cd hostmaster.git # look at the authors in the original repo git fast-export --all | grep -a '^author [^<]* <[^>]*> [0-9][0-9]* [+-][0-9][0-9][0-9][0-9]$' | sed 's/[0-9][0-9]* [+-][0-9][0-9][0-9][0-9]$//' | sort | uniq -c git fast-export --all | perl /home/anarcat/bin/rewrite_authors.pl | ( cd ../../export/hostmaster.git ; git fast-import ) cd ../../export/hostmaster.git # look at the authors in the new repo git fast-export --all | grep -a '^author [^<]* <[^>]*> [0-9][0-9]* [+-][0-9][0-9][0-9][0-9]$' | sed 's/[0-9][0-9]* [+-][0-9][0-9][0-9][0-9]$//' | sort | uniq -c # rinse, repeat until the mapping is right # push repositories online git remote add origin ssh://gitosis@git.aegirproject.org/export/hostmaster.git git push --all git push --tags # repeat with provision
This is the script source, without all the mappings:
#! /usr/bin/perl -w %authors = ( 'anarcat ' => 'Antoine Beaupré ', # ... ); sub remap { my $a = shift; if ($authors{$a}) { return $authors{$a}; } else { return $a; } } while (<>) { s#^author ([^<]+ <[^>]+>) ([0-9]+ [+-][0-9][0-9][0-9][0-9])$#'author ' . remap($1) . ' ' . $2#e; print; }
Migration checklist
- serve exported repositories to migration team done!
- update the repositories with change authors done!
- the great git migration done! git.aegirproject.org is now readonly, commits are pushed to git.drupal.org and releases are performed on drupal.org
- update documentation in handbook in progress!
- repositories locations done
- crash course done
- basic workflow done
- release process needs review - updated for release nodes stuff, will check workflow during the next release
- advanced workflow: branch naming conventions in progress - see 0.4-rc2 release coordination
- migration of the install docs into the handbook work in progress
- update the home page links done - I have changed the link in the body from git.aegirproject.org to drupal.org provision/hostmaster project pages, and pointed the "Get the source" link in the bottom to the install manual instead
- update makefiles and files in docs/* in provision to fetch from git.drupal.org done!
- make at least one other release done - 0.4-rc2 release coordination
- turn off git.aegirproject.org done
Architecture Guide
This section deals with providing coherence and understanding of various aspects of Aegir - what it is, and what it is not.
It tries to explain some of the concepts with which Aegir is developed for and how all the bits hook together.
Aegir design and terminology
@TODO - this document is out of date and needs updating.
This page documents all the different terms used when referring to components of the Hostmaster system, and how the different entities relate to each other.
Front End
The user interface used to administrate your sites. The front end is provided by the Hostmaster install profile, and the Hosting contributed module. It defines a complete Drupal based data model for managing the various aspects of your installation.
Entities
These entities are used primarily in the front end to manage the configuration and data. Upon calling the back end, the system breaks down the properties of these entities to be passed as options for the command line, so the back end only has a flat data structure to work with. During this process, all relationships are automatically retrieved, making it a one step process for the developer.
- Client
- The person or group that runs the site. This information is usually required for billing and access purposes, to assure that only certain people are able to view the information for sites they run. If you do not intend on having more than one client access the system, you will not need to create any additional clients for your purposes.
- Site
- An instance of a hosted site. It contains information relating to the site, most notably the domain name, database server and platform it is being published on. A site may also have several aliases for additional domains the site needs to be accessible on.
- Task
- The mechanism whereby Hostmaster keeps track of all changes that occur to the system. Each task acts as a command for the back end, and contains a full log of all changes that have occurred. If a task should fail, the administrator will be notified with an explanation of exactly what went wrong, and how to fix it.
- Platform
- The file system location on a specific web server on which to publish sites. Multiple platforms can co-exist on the same web server, and need to do so for upgrades to be managed, as this is accomplished by moving the site to a platform hosting an updated release. Platforms are most commonly built for specific releases of Drupal.
- Server
- The server that provides various services.
- Service
- The service that runs on the server. This might be HTTP or a Database server. Each web server hosts one or more platforms, which act as publishing points for the hosted sites. If you are not intending to use Hostmaster in a distributed fashion, you will not need to create additional web servers for your purposes. Most web servers and database servers are on the same machine, but for performance reasons external database servers might be required. It is not uncommon for one database server to be shared amongst all site instances. If you are not intending to use an external database server, or multiple database servers, you will not need to create any additional database servers for your purposes.
- Service type
- The type of service that runs on the server. In the example of 'HTTP' as a service, a service *type* is Apache or Nginx, for example. MySQL is a *type* of service 'Database'.
- Package
- An installable component for a site. Hostmaster keeps tracks of all the modules, themes and install profiles installed across your entire network. These packages most commonly link to their project on drupal.org, but might not in the case of custom modules. Users are not able to create nodes of this type, as they are generated automatically during verify and sync tasks.
- Package Release
- The release of a specific package. Each package on your network has at least one installed release, and each platform (including Drupal itself) may have several releases installed across your network.
- Package Instance
- Where a package release has been installed. A package release may be installed in multiple places on a platform, and sites may have access to multiple versions of packages. This entity also tracks which modules are enabled, and what version of the schema (if any) they have installed.
Task types
These are "hosting commands" that the server can accomplish. These are mapped to back end commands by the task queue processor.
- Verify platform
- Verify that a platform is correctly installed. Automatically created when new platforms are added, to ensure that they are capable of having new sites installed on them. Operates on: platforms
- Import sites
- Import existing sites onto a platform. Operates on: platforms
- Install site
- Install a new site. This is automatically generated when a site node is created. Operates on: sites
- Sync site
- Synchronize changes to the site record with the back end. Automatically generated on editing of the site node. Operates on: sites
- Backup site
- Generate a backup of an existing site. This backup contains everything needed to run the site. Operates on: sites
- Restore backup
- Restore a site to a previous backed up version. Operates on: sites
- Disable site
- Disable a running site. This is commonly used in the case of non-payment from clients. Operates on: sites
- Enable site
- Enable a disabled site. The opposite of Disable. Operates on: sites
This is an incomplete list, and will be updated as more tasks are added
Queues
The front end uses and maintains several queues, which are then processed through the drush.php hosting dispatch
command.
More queues can be added by optional contributed modules, and are often used to provide regularly scheduled back end commands,
such as backing up sites.
- Task Queue
- The primary and most important queue. Whenever a change is made to the data set, the front end creates a "Task" node. These task relate to such things as installing new sites, regenerating the configuration files of sites, verifying that a platform has been correctly configured and importing existing sites on a newly added platform.
- Cron Queue
- This queue is used to manage the regular execution of the cron process required by most sites. It is configurable with a frequency you want all the sites to be cronned within. This is commonly 1 hour, but might be higher or lower. Higher frequency will cause higher system load. This command will iterate through all enabled sites, and batch the cron calls, based on how many sites are available, and how frequently it is running.
- Backup Queue
- This queue operates in the same way as the cron queue, in that it iterates through all installed sites, based on a configurable frequency. This executes the same functionality as requesting a site backup through the user interface. You are able to revert back to a previous backup of your sites.
- Statistics Queue
- This queue operates on the same mechanism as the cron queue, and alows you to retrieve statistics of your installed sites. This will retrieve such metrics as number of registered and active users, and number of posts / comments, which will then be viewable on the front end.
Back End
The back end is a command line interface provided by the Provisioning Framework, of all the tasks it is capable of performing.
It is designed to operate separately from the front end, so that you can have multiple back ends on one or more servers.
The back end handles the server level configuration of the system, such as creating new databases and virtual host records and restarting the apache process for new installations to take effect.
When the queues are processed by the front end, these are turned into calls to the back end, for whichever server they happen to be on. The back end communicates to the front end through a very simple mechanism, whereby it returns a serialized string containing state information and log messages / error return codes.
Aegir Architecture
This wiki page documents the Aegir architecture in schematic form. It is intented to complement a related page on the typical file system structure on an Aegir installation.
Click on image for a larger version.
Aegir file system structure
This page documents the typical file system structure on an Aegir installation. It is intended to complement a related wiki page on Aegir Architecture. The following paths are based on an Aegir 0.4-x installation and assume that all directories are within the /var/aegir folder.
Scripts and Configuration Files
Path | Notes |
---|---|
/backups | site-specific tar balls containing a database dump and folders under path/to/platform/sites/example.com |
/config | |
/config/server_master | |
/config/server_master/apache | |
/config/server_master/apache/conf.d | non-aegir or non-drupal virtual hosts files |
/config/server_master/apache/platform.d | contains .htaccess information for each aegir platform |
/config/server_master/apache/vhost.d | apache virtual host files for aegir platforms |
/config/server_master/apache/vhost.d/aegir.example.com | virtual host file for aegir web site - specifies path to platform directory and site database settings (so that database credentials are not exposed directly in site settings.php file) |
/config/server_master/apache/vhost.d/site-1.com | virtual host file for deployed web site |
/config/server_master/apache/vhost.d/site-2.com | |
/config/server_master/apache/vhost.d/site-3.com | |
/drush | drush folder |
/drush/drush.php | drush script |
/.drush | drush extensions and server, platform and site aliases |
/.drush/drush_make | drush_make project folder - used for building web sites from .make files |
/.drush/provision | provision folder |
/.drush/server_master.alias.drushrc.php | settings for the master server where the main aegir database, hosting platform and aegir site reside |
/.drush/platform_hostmaster.alias.drushrc.php | settings for the hostmaster platform on which the aegir site is based |
/.drush/hostmaster.alias.drushrc.php | settings for the aegir site |
/.drush/platform_platform1.alias.drushrc.php | settings for platform1 on which site-1.com is based |
/.drush/site-1.com.alias.drushrc.php | settings for site-1.com |
Path | Notes |
---|---|
/hostmaster-0.x | hostmaster platform |
/hostmaster-0.x/profiles | |
/hostmaster-0.x/profiles/default | |
/hostmaster-0.x/profiles/hostmaster | hostmaster profile |
/hostmaster-0.x/profiles/hostmaster/modules | |
/hostmaster-0.x/profiles/hostmaster/modules/admin_menu | |
/hostmaster-0.x/profiles/hostmaster/modules/hosting | hosting module |
/hostmaster-0.x/profiles/hostmaster/modules/install_profile_api | install_profile_api – facilitates provisioning of sites based on a non-default profile |
/hostmaster-0.x/profiles/hostmaster/modules/jquery_ui | |
/hostmaster-0.x/profiles/hostmaster/modules/modalframe | |
/hostmaster-0.x/profiles/hostmaster/themes | |
/hostmaster-0.x/profiles/hostmaster/themes/eldir | eldir theme – provides Aegir front end look and feel |
/hostmaster-0.x/profiles/hostmaster/hostmaster.profile | profile file – used in site provisioning to configure a drupal database |
/hostmaster-0.x/profiles/hostmaster/hostmaster.make | make file – used to include modules, themes, libraries etc. from various sources |
/hostmaster-0.x/modules | |
/hostmaster-0.x/themes | |
/hostmaster-0.x/sites | |
/hostmaster-0.x/sites/aegir.example.com | aegir web site folders |
Deployed Platforms
Note: the directory /platforms is optional but can be useful to separate deployed platforms from directories for scripts, config files and hostmaster platform.
Path | Notes |
---|---|
/platforms | |
/platforms/platform-1 | |
/platforms/platform-1/profiles | |
/platforms/platform-1/profiles/default | |
/platforms/platform-1/profiles/custom-profile | |
/platforms/platform-1/profiles/custom-profile/modules | |
/platforms/platform-1/profiles/custom-profile/themes | |
/platforms/platform-1/profiles/custom-profile/custom.profile | profile file – used in site provisioning to configure a drupal database |
/platforms/platform-1/profiles/custom-profile/custom.make | make file – used to include modules, themes, libraries etc. from various sources |
/platforms/platform-1/modules | |
/platforms/platform-1/themes | |
/platforms/platform-1/sites | |
/platforms/platform-1/sites/site-1.com | |
/platforms/platform-1/sites/site-1.com/modules | |
/platforms/platform-1/sites/site-1.com/themes | |
/platforms/platform-1/sites/site-1.com/files | |
/platforms/platform-1/sites/site-1.com/settings.php | site-specific drupal configuration file |
/platforms/platform-1/sites/site-1.com/drushrc.php | site-specific aegir-specific configuration file |
/platforms/platform-1/sites/site-2.com | |
/platforms/platform-1/sites/site-3.com | |
/platforms/platform-1/sites/site-n.com | |
/platforms/platform-2 | |
/platforms/platform-3 | |
/platforms/platform-n |
Platform permissions
Path | Directory |
File |
Notes |
---|---|---|---|
./* |
aegir:aegir drwxr-xr-x |
aegir:aegir -rw-r--r--
|
The webserver has no business writing or moving files in the Drupal codebase. |
Inside ./sites/example.com: | |||
drushrc.php |
aegir:aegir -r--------
|
Web server shouldn't be able to read drushrc.php, it's not a component of the Drupal site. | |
settings.php |
aegir:www-data -r--r-----
|
Web server can read this file, but otherwise tight control over this file which can contain sensitive information. | |
libraries modules
themes
|
aegir:aegir drwxrwsr-x
|
aegir:aegir -rw-r--r--
|
Because of the sticky bit (s) on the parent directory, each child directory will inherit attributes of the parent. The attribute that is consistently inherited is the group. This means that a developer in the "aegir" group can add files which will retain the "aegir" group ownership of the parent directory. |
files private
|
aegir:www-data drwxrws---
|
Aegir sets these directories with a sticky bit (s) so that under certain conditions new folders and files will inherit parent permissions. There are only a few cases where this happens though. | |
files/* private/*
|
www-data:www-data drwxr-sr-x
|
www-data:www-data -rw-r--r--
|
The permissions shown here are how files and directories created by www-data will look. When verifying a platform, Aegir won't "correct" these files and directories to match the parents.
If you have trouble with permissions/ownership on these directories, you can safely run the following commands (in this case on the files directory):
# chown -R aegir:www-data /path/to/site/files/* |
Provision - the Aegir backend.
Each of the major entities managed by Aegir (namely servers, platforms and sites) are represented by nodes in the front end, which in turn generate Drush 'named contexts' in the backend. Named contexts are similar to what Drush 3.0 refers to as 'site aliases', but are more structured and cover more than just sites.
Whenever you create or edit a node on the front end, Aegir will create or update the named context on the backend, and any tasks run on the entity will be run on that named context, ie:
drush @server_master provision-verify
All of the actual work related to managing your system is done by the backend, and the front end primarily serves as a way to manage these aliases. You can very easily use the backend on it's own if you so choose.
Creating or updating named contexts.
Contexts are created and updated through the provision-save command. The format of the command is :
drush provision-save @contextname --context_type=type --property=value --other_property=value
Every time this command is called, it will update the named context and only modify the properties that have been specified. It is important to note that the provision-save command never has a site alias or context name before the command, but requires you to specify which context you are modifying as the first argument.
Deleting named contexts.
To delete an existing named context, you simply need to pass the 'delete' option to the provision-save command, ie:
drush provision-save @contextname --delete
Using named contexts
Named contexts are used in 3 ways in provision: as aliases, arguments or options.
Aliases : Aliases appear to the left of the drush command, and are making use of the Drush 'site alias' functionality.
# Verify the site represented by @example.com
drush @example.com provision-verify
Options : Certain options to provision-save require you to pass the alias of another named context of a specific type, and uses this information to build relationships between objects.
# generate a new site context in the directory represented by @platform_test
drush provision-save @example.com --context_type=site --uri=example.com --platform=@platform_example
Arguments : A select few provision commands require you to pass the alias of another specific type of named context as an argument.
# Migrate the site to the platform represented by @platform_newversion
drush @example.com provision-migrate @platform_newversion
Context types
There are (currently) 3 different entities represented as named contexts in the aegir backend. You determine which type of entity you are working with by specifying the context_type option to the provision-save command. You dont need to specify it every time, but it is best to always explicitly specify it, just in case.
Because the namespace of all contexts is shared, they each have different naming conventions to avoid collisions. There are certain named contexts of each type which are always present in the system.
Each of the context types have different semantics and properties, and certain provision commands will only operate on certain types of entities.
Server contexts
Naming guideline : Server with a hostname of server.example.com becomes @server_serverexamplecom.
Special contexts : @server_master always refers to the local server where aegir is installed.
Required properties :
- remote_host : This should always be set to the FQDN of the server being managed.
Services : Servers are the only entities which have services associated to them. Services are things such as http, db or dns.
Each service may select one service_type, which chooses which implementation to load for this service. Examples of these are the 'mysql' implementation of the 'db' service, or the 'apache' and 'apache_ssl' implementations of the http service.
All services that are found by Aegir are enabled, but unless you specifically select one of the implementations, the 'null' service type is loaded. Each service type has additional required properties and values that can be passed to it, but describing them all is outside of the scope of this document.
Important commands :
- provision-verify : Verify that the configuration is correct and fix them if they aren't.
Example :
# Enable a new remote server hosting sites with apache on port 8080
drush provision-save @server_webexamplecom --context_type=server \
--remote_host=web.example.com \
--http_service_type='apache' \
--http_port=8080
# Verify that the settings are correct
drush @server_webexamplecom provision-verify --debug
Platform contexts
Platforms are how aegir refers to a specific copy of Drupal checked out on a file system. Platforms are hosted on web servers and sites are hosted on platforms.
Naming guideline : A platform in /var/aegir/codebase-6.1 becomes @platform_codebase61 , but this is not strictly enforced. We only use that naming convention when we do not have a node representing the platform available. If a node is available, we use the title of the node to generate the context name.
Basically, you can name your platform anything but it should still be prefixed with 'platform_'.
Special contexts : There are no special contexts per se, but the hostmaster-install and hostmaster-migrate commands generate aliases for the hostmaster platform in the form of '@platform_hostmaster$version'. If the front end is present, one or more of these contexts are usually available too.
Required properties :
- root : The absolute path to a Drupal directory, generally in the form /var/aegir/platformdir.
Optional properties :
- web_server : The name of a server context that provides the http service. The default is to @server_master
- makefile : The absolute path or url to a drush make makefile. If the root directory does not exist and the makefile property does, provision will call drush_make to attempt to create the directory.
Important commands :
- provision-verify : This will ensure that all the files are the correct permissions, and are all present on the correct remote servers. Also rebuild package database. Run this command whenever the contents of the platform changes.
Example :
# New open atrium platform in /var/aegir/atrium-head that will be built from a remotely hosted
# makefile and published on an external web server.
drush provision-save @platform_atriumhead --context_type=platform \
--root=/var/aegir/atrium-head \
--web_server=@server_webexamplecom \
--makefile='http://omega8.cc/dev/atrium_stub.make.txt'
# Verify settings and generate the directory with drush make, pushing to the remote server after.
drush provision-verify @platform_atriumhead
Site contexts
Naming guidelines : A site with the url 'site1.example.com' becomes @site1.example.com. The context name is always the url.
Special contexts : @hostmaster always refers to the site running the hostmaster front end. This helps with debugging and makes it more easily scriptable.
Required properties :
- uri : The FQDN of the site. should match the context name.
- platform : The name of a platform context that the site will be hosted on.
Optional properties :
- db_server : The name of a server context which provides the db server. defaults to @server_master.
- client_email : The email address to send the login information for the admin user.
- profile : The installation profile the site is running. dependent on the platform supporting it.
- language : The ISO language code of the site. dependent on the platform and profile supporting it.
Important commands :
Almost all of the provision commands are related to sites, so this is just the very core of them. Check drush help for more.
- provision-install : Install the site represented by the context. Just creating the context doesn't install the site yet.
- provision-verify : Verify the site is running correctly, and ensure the files are all published to the correct servers.
- provision-delete : Delete the site and all it's information. The context will still be there.
Example :
# Set up a new site to be installed on the open atrium platform (hosted on a remote server),
# using an external database server , with the openatrium profile and localize it into spanish.
drush provision-save @atrium.example.com --context_type=site \
--uri=atrium.example.com \
--platform=@platform_atriumhead \
--db_server=@server_dbexamplecom \
--client_email=client@example.com \
--profile=openatrium \
--language=es
# Install the new site to the specifications requested above.
drush @atrium.example.com provision-install --debug
# Verify the site to make sure everything is ok.
drush @atrium.example.com provision-verify --debug
Importing contexts into the front end
Hostmaster provides a drush command which will attempt to generate or updates front end nodes that match your defined contexts. To import a context into the front end, just run :
drush @hostmaster hosting-import @contextname
This will import that context, and every context it depends on into the front end. This is useful for integration with other systems such as the puppet-aegir module for Puppet, which can add newly configured servers directly to the front end.
Messaging systems evaluation
So we've been talking seemingly forever about possible queuing system instead of our crappy "PHP/Vixie Cron based" one, but we have yet to document pro/cons of possible alternatives. Here are the goods (or the beginning of em).
Note that this document was written before (in november 2009) the current multiserver implementation was designed and completed.
Current design
Tasks are basically serialized in a MySQL database and polled at regular interval by a cron job, running drush hosting dispatch
, which bootstraps Drupal, fetches the object, and fires up provision commands.
The possibility here is to have another dispatcher that would always run, and maybe not even talk to the mysql database, to fire up the provision commands.
Another requirement is that the queuing system may be network-aware. We want multi-server support and that is a critical issue for the next release (0.4 at the time of writing). One thing to consider is how we pass files around...
Roll our own
Pro:
- full control over the API, functionality and whatnot (ie. function of manpower invested)
Con:
- rolling our own is hard! let's go shopping!
Meta: AMQP vs JMS
So yeah, there's a standard in development for all this stuff: http://www.AMQP.org/ . There is also a Drupal module being developed to work with it: http://drupal.org/project/amqp
JMS on the other hand is the "Java Messenging Service", that creates a language- (as opposed to network-) based API.
Maybe we should just not care about which messaging system you use in the backend and allow our frontend to talk to any backend, with the stupid cronjob mysql poller as a default...
Gearman
Pros (mostly out of the home page):
- supports PHP in frontend and backend
- Multi-language - There are interfaces for a number of languages, and this list is growing. You also have the option to write heterogeneous applications with clients submitting work in one language and workers performing that work in another.
- Flexible - You are not tied to any specific design pattern. You can quickly put together distributed applications using any model you choose, one of those options being Map/Reduce.
- Fast - Gearman has a simple protocol and interface with a new optimized server in C to minimize your application overhead.
- Embeddable - Since Gearman is fast and lightweight, it is great for applications of all sizes. It is also easy to introduce into existing applications with minimal overhead.
- No single point of failure - Gearman can not only help scale systems, but can do it in a fault tolerant way.
- Production ready - ran on digg, yahoo, livejournal, ...
- Gearman uses an efficient binary protocol and no XML. There's an a line-based text protocol for admin so you can use telnet and hook into Nagios plugins.
- Persistent queues are available using mysql, memcached or sqlite
- There's pretty good PHP support and it's easy to get up and running quickly.
Cons:
- "somewhat higher latency, signal-and-pull architecture"
- The system makes no guarantees. If there's a failure the client is told about the failure and the client is responsible for retries.
- gearman-specific binary/network protocol
- Notice how sixapart.com maintains both gearman and the shwartz, below
OHLOH:
- Mostly written in Perl
- Increasing year-over-year development activity
- Established codebase
- Few source code comments
Resque
Pro:
- redis-based (therefore somehow language agnostic deep in there, but certainly network-centric)
- github's (aka: it's got to work at least somewhere in production)
Con:
- ruby-centric
- I haven't actually looked at the thing or tested it
Active MQ
Apache ActiveMQ:
Pros:
- Supports a variety of Cross Language Clients and Protocols from Java, C, C++, C#, Ruby, Perl, Python, PHP
- Supports many advanced features such as Message Groups, Virtual Destinations, Wildcards and Composite Destinations
- Supports pluggable transport protocols such as in-VM, TCP, SSL, NIO, UDP, multicast, JGroups and JXTA transports
- REST API to provide technology agnostic and language neutral web based API to messaging
- Persistent
Cons:
- uses too many acronyms on the frontpage (really.)
- uses a lot of Java jargon
- production-ready?
OHLOH:
- Mostly written in Java
- Large, active development team
RabbitMQ
Pros:
- "high reliability, availability and scalability along with good throughput and latency performance that is predictable and consistent"
- "based on the emerging AMQP standard"
- in debian
- Drupal module - AMQP
- apparently very fast, according to dellintosh
Cons:
- erlang? do you know erlang? (Arguably not really necessary, since all publishers and consumers can be written in other languages that have AMQP support)
- Mostly written in Erlang
- Large, active development team
- Well-commented source code
Open Messenging Queue
https://mq.dev.java.net/overview.html
Pros:
- SOAP/HTTP interface
- Scalable
Con:
- Java/C specific
The Schwartz
Pros:
- Perl (come on, perl is everywhere... isn't it?)
- "reliable job queue"
- has retry
- "lightweight"
Cons:
- Perl (aka write only)
- "library, so some assembly required"
- "light on documentation"
- Notice how sixapart.com maintains both the shwartz and gearman, above
Open AMQ
Pros:
- Multi-language (but not PHP)
- "learn in a day or so" then "another day or two for wireAPI", development will take "some weeks"...
- "remote admin tools, one-line failover, instant federation, protection against slow clients, detailed logging"
- AMQP standard
- "Linux, AIX, Solaris, Mac OS/X, other UNIX"
Cons:
- "for C/C++ and JMS"
- Mostly written in C
- Well-commented source code
- Short source control history
- Only a single active developer
- Apache Software License may conflict with GPL
- Apache License 2.0 may conflict with GPL
Beanstalkd
http://kr.github.com/beanstalkd/
Drupal Module for Drupal 6 and 7 is at http://drupal.org/project/beanstalkd
Pros: * used in production ("causes" application in facebook)
Cons:
- C
http://www.ohloh.net/p/beanstalkd
- Mostly written in C
- Large, active development team
- Few source code comments
SimpleMQ
http://code.google.com/p/simple-mq/
Pros:
- persistent or in-memory
Cons:
- java-only.
Zero MQ
Pros:
- fast
- lightweight messaging implementation.
- supports different messaging models.
- already very fast. We're getting 13.4 microseconds end-to-end latencies and up to 4,100,000 messages a second today.
- very thin. Requires just a couple of pages in resident memory.
- provides C, C++, Java, Python, .NET/Mono, Ruby, Fortran, COBOL, Tcl, Lua and Delphi language APIs.
- supports different wire-level protocols: TCP, PGM, AMQP, SCTP.
- runs on AIX, FreeBSD, HP-UX, Linux, Mac OS X, OpenBSD, OpenVMS, QNX Neutrino, Solaris and Windows.
- supports i386, x86-64, Sparc, Itanium, Alpha and ARM microarchitectures.
- fully distributed: no central servers to crash, millions of WAN and LAN nodes.
Cons:
- no PHP whatsoever
- production?
OHLOH:
- Mostly written in C++
Jenkins
There's a good vibe around using Jenins (formerly known as Hudson) in place of cron for those kind of things. In our scenario, Aegir would queue up jobs to Jenkins, which would take care of dispatching them to remote servers.
Pros:
- supports tons of different notification modes
- scales well
- supports remote servers
Cons:
- application-specific (unclear?) protocol to queue up tasks
- Java (so a bit heavy)
OHLOH: * has had 63,860 commits made by 708 contributors * representing 893,049 lines of code * is mostly written in Java * with an average number of source code comments * has a well established, mature codebase * maintained by a very large development team * with decreasing year-over-year commits
Others to evaluate?
- Amazon's SQS. The latencies for this service tend to be high and variable so it may not be appropriate for all tasks.
- Spread Queue ([[http://search.cpan.org/~jmay/Spread-Queue-0.4/Queue.pod|alpha software]]
- Starling (apparently, twitter ditched starling for scala)
- Houston From Zivtech, not much documentation yet
Others mentionned here: ActiveMessaging, BackgroundJob, DelayedJob, and Kestrel.
Requirements
All the above must have:
- Open Source - It's free as in beer and speech
Provision contexts
This section is currently being written, so its probably going to be full of errors, and possibly not that useful. Feel free to edit.
I'm probably going to pop a nice table of contents here, for now look to the right...
Purpose
Contexts are simply a way for Aegir to store certain pieces of information that it thinks may be useful later. They are named so that they can be referred to easily.
Every server, platform and site in Aegir has an associated context. Each one of these contexts stores information about the associated entity. So, for example, a site context stores the base URI of the site. Later, the code responsible for informing the web server of the site's existence can look at this context and determine the requested URI.
In theory contexts allow most of the back-end commands in Aegir to operate just on the name of the context. This is an important concept: the alternative would be to try to guess everything that the command being invoked might want to know and pass it in as a series of command line arguments.
For example, the command that Aegir uses to clone a site just needs to be given the context of the site to clone and the platform to clone it to. From there the code can work out where the destination platform actually is, what servers it needs to talk to, and with what credentials.
Drush contexts
Note that Drush also has the concept of a context, but this is not to be confused with Aegir contexts which are different. Aegir contexts are closer to, and stored as, Drush 'aliases'.
Implementation
Contexts are always named to start with a '@' symbol, except for in the filename of the file in which they are stored.
Contexts in Aegir are stored as a simple array of data in a file within the backend of Aegir. However this array of data is accessed and modified with a set of objects and accessors. Here is an example of what a context file looks like:
<?php
$aliases['hostmaster'] = array (
'context_type' => 'site',
'platform' => '@platform_hostmaster',
'server' => '@server_master',
'db_server' => '@server_localhost',
'uri' => 'aegir.example.com',
'root' => '/var/aegir/hostmaster-0.4-beta2',
'site_path' => '/var/aegir/hostmaster-0.4-beta2/sites/aegir.example.com',
'site_enabled' => true,
'language' => 'en',
'client_email' => 'aegir@example.com',
'aliases' =>
array (
),
'redirection' => false,
'profile' => 'hostmaster',
);
These files are stored in the /var/aegir/.drush
directory on the master Aegir server.
In the above example we can see the properties of the context, like the 'uri' of the site, and the 'profile' that the site is running.
There are also a number of more interesting properties in the example, note the 'db_server' property, which has a value that is another context. We shall see later that these properties are special, and allow developers to easily access functions of the 'db_server' without needing to care which db server they are talking to.
Contexts are used within drush commands as objects, which subclass provision_context
. This allows much more flexibility and cleverness, though can make them very confusing to use sometimes! As a developer you will only need to worry about the context objects, as Aegir handles storing them in the files for you. But it is important to note that each context must be representable in a flat text file (or be prepared to do some serious leg-work) by that I mean you probably don't want to be storing massive amounts of relational data in them, use a database for that!
Using contexts in your code
Getting the current context is really easy, just call the 'd
' accessor:
<?php
$current_context = d();
?>
d()
would return the site context. The values of the context are accessible as simple properties of this object.
So, suppose a verify task is started for the main Aegir frontend, which has a context that is always called '@hostmaster', the drush command invocation would look like this:
drush @hostmaster provision-verify
Then the drush command for provision-verify
could do this:
<?php
function drush_provision_verify() {
$context = d();
drush_print('Passed context of type: ' . d()->type);
}
?>
The d
accessor is not to be feared! It is used all over the place in Aegir and if you're not sure what context you're going to get then you can always print d()->name
and you'll get the name of the context that you're dealing with.
Loading a named context
You can pass an optional argument to the d
accessor of the name of the context that you want. So, if you feel compelled to access a property about the main Aegir frontend site anywhere you can do this:
<?php
$hostmaster_context = d('@hostmaster');
?>
though most of the time you'll be dealing with the current context, and contexts that it references.
'Subcontexts'
Here's an example context:
<?php
$aliases['hostmaster'] = array (
'context_type' => 'site',
'platform' => '@platform_hostmaster',
'server' => '@server_master',
'db_server' => '@server_localhost',
'uri' => 'aegir.example.com',
'root' => '/var/aegir/hostmaster-0.4-beta2',
'site_path' => '/var/aegir/hostmaster-0.4-beta2/sites/aegir.example.com',
'site_enabled' => true,
'language' => 'en',
'client_email' => 'aegir@example.com',
'aliases' =>
array (
),
'redirection' => false,
'profile' => 'hostmaster',
);
Notice that some of the properties are the name of contexts, such as 'db_server' which has a value of '@server_localhost'.
Suppose now that I have some code that wants to get a property of the db_server associated with the @hostmaster context, I can simply do this:
<?php
$db_server_context = d('@hostmaster')->db_server;
// $db_server_context is now a fully populated context object, not the string '@server_localhost'
drush_print('DB server on port: . $db_server_context->db_port);
?>
The provisionContext
object will return full context objects for properties that store the names of other contexts. The how and why of determining which to return as strings and which to return as a context object will be covered later. It is entirely possible to have the name of a context stored as a property in a context, that when accessed returns the string, rather than the context named in the string.
Properties and services
Services are the way in which new properties are added to contexts, in fact if you wish to indicate you want to store additional properties in a context, this must be done by a service. There is no other way to store data in a context. One might assume that you can just ask provision to save an additional value on a context, but this will not work, and you'll probably get very frustrated indeed!
For example, the provisionService_pdo
service adds a 'master_db' property to the server context, it does this in a method thus:
<?php
function init_server() {
parent::init_server();
$this->server->setProperty('master_db');
}
?>
This means that the 'master_db' property can now be set in a provison-save drush command, and will be persisted when the context is saved.
A service may also define properties on the context as actually representing another context, so that you may access that subcontext directly. You can see an example of using this subcontext in the 'Implementation' section of this Provision Contexts guide, but the an example of how you let provision know that a property name is not just a string, but a named context is with a method on the service:
<?php
/**
* Register the http handler for platforms, based on the web_server option.
*/
static function subscribe_platform($context) {
$context->setProperty('web_server', '@server_master');
$context->is_oid('web_server');
$context->service_subscribe('http', $context->web_server->name);
}
?>
provisionService_http
is being asked to 'subscribe' to a platform, it sets a property on the context, as above, but additionally uses the is_oid()
method to indicate that the property is a named context. It then uses that immediately when it gets the name of the web_server context: $context->web_server->name
. We'll worry about what 'subscribing' to a platform means later,. Using services from contexts
This is stub page that will describe the way that you can use the service()
method on a context.
Provision services
We should add some documentation about Provision services, what they are and how they work and interact with contexts.
UNIX group limitations
While working on the security enhancements, I stumbled upon a few fundamental UNIX limitations with groups. These limitations similarly apply to GNU/Linux.
1. Group name length limits
A group name (e.g. dialout
) has a length limit, usually hardcoded in the operating system. This limit varies across different UNIX systems and implementations.
- NIS: 8 characters (source)
- HPUX: 8 characters, 16 starting from HPUX 10 (source, source)
- AIX: 8 characters, 255 for AIX 5.3 and above (source)
- Debian: 16 characters (source, defined in shadow at compile-time)
- Redhat: 16 characters, 32 starting from 2005 (source, source, source)
- Solaris: 8 characters? (source)
- Active Directory: 11 or 64 characters (source unclear)
In general, we can probably assume that 16 characters is a safe limit, and that's what's being used in hosting_client_sanitize.
2. Group membership limits
A UNIX user has a primary group, defined in the passwd
database, but can also be a member of many other groups, defined in the groups
database. UNIX has a hardcoded limit of 16 groups a user can be a member of (source). This means that if we translate Aegir clients into UNIX groups and you have access to multiple Aegir clients, you'll be able to access only 16 of those clients at a time. Rather annoying.
The problem is related with NFS, or rather AUTH_SYS
(source), which has an in-kernel data structure where the groups a user has access to is hardcoded as an array of 16 identifiers (gids
to be more precise, see this post for details). Even though Linux now supports 65536 groups, it is still not possible to operate on more than 16 from userland, through NFS - on a regular ext3 filesystem, it works without problems (tested in Debian Squeeze).
Solutions for this are not obvious. The blog post on sun.com proposes a technical enhancement to OpenSolaris. It's unclear whether that fix was implemented. This SAGE post explains that using GSS_API
(used in NFSv4) should (should!) resolve the issue.
This more thorough analysis of the problem is more direct, to shamelessly quote it:
- Use NFSv4
- Use RPCSEC_GSS authentication
- Use ACLs
As mentionned above, this only applies to NFS mounts. NFSv4 should (should!) be simple enough and available on Linux. It is also unclear exactly how to use RPCSEC_GSS
(is that Kerberos?). As for ACLs, we are fully supporting this now with the provision ACL extension, so that shouldn't be an issue.
Running drush on Aegir sites as a regular user
N.B. The ACL-based approach, described below, has been implemented as an extension to Provision in http://drupal.org/project/provisionacl.
What's the problem here?
As things stand, you cannot run tasks as a regular UNIX user on your Aegir Drupal sites:
anarcat@marcos:~$ drush @hostmaster cc all
A Drupal installation directory could not be found [error]
The drush command '@hostmaster cc all' could not be found. [error]
The reason the site is not found is because the alias cannot be loaded. If we change our home directory to find the alaises, we then get permission problems:
anarcat@marcos:~$ env HOME=/var/aegir drush @hostmaster cc all
include(/var/aegir/.drush/platform_hostmaster.alias.drushrc.php): failed to open stream: [warning]
Permission denied sitealias.inc:433
include(): Failed opening '/var/aegir/.drush/platform_hostmaster.alias.drushrc.php' for [warning]
inclusion (include_path='.:/usr/share/php:/usr/share/pear') sitealias.inc:433
include(/var/aegir/.drush/hostmaster.alias.drushrc.php): failed to open stream: Permission [warning]
denied sitealias.inc:433
include(): Failed opening '/var/aegir/.drush/hostmaster.alias.drushrc.php' for inclusion [warning]
(include_path='.:/usr/share/php:/usr/share/pear') sitealias.inc:433
A Drupal installation directory could not be found [error]
The drush command '@hostmaster cc all' could not be found. [error]
... so that doesn't seem to be a good approach, although one could just fix the permissions on the alias files for the site (which don't contain the DB password, oddly enough) and platforms (which have been safe since the "server" seperation):
cd /var/aegir/.drush
chmod a+r ls *alias* | grep -v 'server'
Then we end up with another problem:
Cannot open drushrc "/var/aegir/hostmaster-HEAD/sites/aegir.anarcat.ath.cx/drushrc.php", [warning]
ignoring.
Cannot open drushrc "/var/aegir/hostmaster-HEAD/drushrc.php", ignoring. [warning]
Cannot open drushrc "/var/aegir/hostmaster-HEAD/sites/aegir.anarcat.ath.cx/drushrc.php", [warning]
ignoring.
include_once(/var/aegir/hostmaster-HEAD/sites/aegir.anarcat.ath.cx/settings.php): failed to [warning]
open stream: Permission denied bootstrap.inc:400
include_once(): Failed opening './sites/aegir.anarcat.ath.cx/settings.php' for inclusion [warning]
(include_path='.:/usr/share/php:/usr/share/pear') bootstrap.inc:400
Cannot modify header information - headers already sent by (output started at [warning]
/usr/share/drush/includes/drush.inc:820) install.inc:618
Cannot modify header information - headers already sent by (output started at [warning]
/usr/share/drush/includes/drush.inc:820) install.inc:619
Drush command could not be completed. [error]
... and this is really the core of the issue here: we need to allow more than one user to access those files, and aegir must be able to fix those permissions when the site is created/verified.
So to sum up:
- a regular user doesn't have permissions to access the drushrc.php and settings.php files
- the aliases are not accessible unless you change your home directory (note that -i doesn't work either, for some reason)
Hackish solution: add user to more groups
Miguel indicates here that files should be writable by the Aegir group, and, if we start with this idea, we can actually get pretty far. The drushrc.php file is owned by the Aegir group, so adding yourself to that group and fixing up the file to be group-readable actually goes a long way. You also need to add the user to the www-data group since drush expects to be able to open the settings.php file also:
adduser anarcat aegir
adduser anarcat www-data
chmod g+r drushrc.php
With the above changes, we can actually run drush commands!
anarcat@marcos:~$ env HOME=/var/aegir drush @hostmaster cc all
Cannot open drushrc "/var/aegir/hostmaster-HEAD/drushrc.php", ignoring. [warning]
WD php: include(/var/aegir/.drush/server_localhost.alias.drushrc.php): failed to open stream: [error]
Permission denied in /usr/share/drush/includes/sitealias.inc on line 433.
WD php: include(): Failed opening '/var/aegir/.drush/server_localhost.alias.drushrc.php' for [error]
inclusion (include_path='.:/usr/share/php:/usr/share/pear') in
/usr/share/drush/includes/sitealias.inc on line 433.
WD php: include(/var/aegir/.drush/server_master.alias.drushrc.php): failed to open stream: [error]
Permission denied in /usr/share/drush/includes/sitealias.inc on line 433.
WD php: include(): Failed opening '/var/aegir/.drush/server_master.alias.drushrc.php' for [error]
inclusion (include_path='.:/usr/share/php:/usr/share/pear') in
/usr/share/drush/includes/sitealias.inc on line 433.
include(/var/aegir/.drush/server_localhost.alias.drushrc.php): failed to open stream: [warning]
Permission denied in /usr/share/drush/includes/sitealias.inc on line 433.
include(): Failed opening '/var/aegir/.drush/server_localhost.alias.drushrc.php' for inclusion[warning]
(include_path='.:/usr/share/php:/usr/share/pear') in /usr/share/drush/includes/sitealias.inc
on line 433.
include(/var/aegir/.drush/server_master.alias.drushrc.php): failed to open stream: Permission [warning]
denied in /usr/share/drush/includes/sitealias.inc on line 433.
include(): Failed opening '/var/aegir/.drush/server_master.alias.drushrc.php' for inclusion [warning]
(include_path='.:/usr/share/php:/usr/share/pear') in /usr/share/drush/includes/sitealias.inc
on line 433.
'all' cache was cleared [success]
We get a bunch of warnings, because provision can't load the server aliases (for a good reason - they contain DB passwords), but the command runs fine! In fact, if we want, we can avoid the luxury of using the aliases and just run the drush command in the site directory:
anarcat@marcos:aegir.anarcat.ath.cx$ drush cc all
Cannot open drushrc "/var/aegir/hostmaster-HEAD/drushrc.php", ignoring. [warning]
'all' cache was cleared [success]
That warning can be removed by simply fixing the permissions on the drushrc.php file to be group-readable.
However, the problem with that approach is that the user has access to all sites, from there on. No regard to what client the user has access to. Obviously, this can work for a dedicated server (and it would need patches to provision so that drushrc.php files and optionnally aliases are group-readable) but it will not work for shared hosting, which is my prime objective here.
Patches necessary:
- group-readable drushrc.php (changes in provision)
- add users to the aegir and www-data groups manually
- optional: group-readable site aliases (so that users can access the aliases)
Issues:
- all or nothing: the user has access to all sites or none as we have a single group
Traditionnal UNIX permission-based approach
In order for this to work really correctly, we would need the following permissions setup. Instead of having the typical following:
-r--r----- 1 aegir aegir 55456 16 mar 13:35 drushrc.php
drwxrws--- 10 aegir www-data 4096 16 mar 13:34 files
drwxrwsr-x 2 aegir aegir 4096 16 mar 13:34 modules
-r--r----- 1 aegir www-data 3276 16 mar 13:35 settings.php
(I am skipping libraries, themes and private directories for simplicity - libraries and themes are the same as modules and private is the same as files.)
... We would need to have the following:
-r--r----- 1 aegir client 55456 16 mar 13:35 drushrc.php
drwxrws--- 10 www-data client 4096 16 mar 13:34 files
drwxrwsr-x 2 aegir client 4096 16 mar 13:34 modules
-r--r----- 1 www-data client 3276 16 mar 13:35 settings.php
That is, all files and directories are readable by the "client" group (which allow inter-client isolation) and some would be readable or writable by the webserver (which allows for uploads and the webserver to read the settings.php and so on).
In this scenario, the webserver has access to all sites all the time: it can read/write uploaded files of other modules, but, because of database cloaking, it can't access other sites' databases. If we want to fix this we would have to go with ACLs.
Note that this setup doesn't require the webserver running any special patch or daemon to run the PHP process as a normal user: it still runs as the www-data user.
Patches necessary:
- patches mentionned above (group-readable drushrc and aliases)
- creating the client group and adding users to it
- adding the aegir user to the client group
- patching provision so it changes the group of some files to the client group
- patching provision so it changes the owner of some files to the www-data user (this actually requires root: ouch!)
Issues:
- requires aegir to run as root
- aegir need to be a member of every group
The ACL approach
This is based on this excellent comment from malclocke.
This is one of the simplest and elegant approach, as it:
- doesn't require root access
- fits well with the n to n client access list model of aegir
- works without having to change the existing permissions (so would fit well in a contrib module)
To configure ACLs, you need to add the option to your mountpoint in fstab and remount the mountpoint (or reboot):
apt-get install acl
mount -o remount,acl /var # and edit /etc/fstab to add the acl option to the mountpoint
The rest is fairly simple: we just need to add access to the required groups to the drushrc and settings.php files:
setfacl -m g:client:r-- settings.php drushrc.php
... where "client" is the client group.
Optionnally, we can also give access to the alias files the same way, but we end up with the same warning on the server level files.
Note: this is a good introduction to ACLs, in Redhat
Patches necessary:
- make platform-level drushrc readable by all (or add an ACL for each client with access to the platform..)
- patch provision to add relevant ACLs on verify and install, to drushrc.php and settings.php:
- system-level configuration of mountpoints and extra package
- creating the client group and adding users to it
Issues:
- requires system-level modifications
- NFS has not supported ACLs very well traditionnally (see the NFSv4 implementation notes and the issues page for ACLs for more information) - but according to my tests (vanilla Debian squeeze), it just works
Other similar research
This is actually a well-discussed topic in the community, although nobody came up with a clean and complete solution for this.
- Brilliant comment from mig5 explaining group-ownership of key files
- A similar solution proposed here actually shows which chunks of code need to be modified in 0.4-alpha6
- Aegir permissions best practices thread - mentions everything from the "aegir group" to ACLs
- My original research into this problem
Extending Aegir
Aegir is designed to be easily extendable by developers. As it is made with Drupal and Drush, it is made of the hooks and command you know and love. If you are a user or admin looking to deploy contrib modules, you should look into the contrib modules list and user documentation instead.
What can I extend exactly?
These extensions may come in the form of
- Adding new tasks to be performed against sites
- Adding new services or implementations of service types (postgres for the DB service, for example)
- Overriding or hooking into existing forms such as the site form, to send extra data to the backend
- Using APIs to inject bits of configuration into configuration files such as settings.php and vhosts.
This area is be devoted to teaching you how to extend and develop for Aegir to encourage contributions to the Aegir project or to help you modify Aegir to suit your unique use case.
Aegir API documentation
The inline documentation is a good start to understand the various hooks and internals that allow you to extend and customize Aegir to your liking. The documentation is rendered on api.aegirproject.org daily.
See also the developer cheat sheet.
Please submit any suggestions or bug reports to the Aegir Project issue queue of your choice, under the "documentation" component.
Example modules
The contrib modules page documents all known Aegir extensions in existence. There are also specific developer documentation pages below.
Contrib developer documentation and roadmaps
Contrib developers are free to use this space to document the internals of their modules or their roadmaps. There is already this documentation:
- E-Commerce Integration Roadmap (uc_hosting)
Aegir developer cheat sheet
1. Locations and paths
- Site URI: d()->uri; // ex: $base_url = 'http://' . d()->uri;
- Drupal root: d()->site // returns: /var/aegir/platform/drupal-6.20/
- Site path: d()->site_path // returns: /var/aegir/platform/drupal-6.20/sites/example.org/
2. File permissions and ownership
$ctools_path = d()->site_path . '/files/ctools'; provision_file()->chmod($ctools_path, 0770, TRUE) ->succeed('Changed permissions of@path
to @perm') ->fail('Could not change permissions@path
to @perm') ->status(); provision_file()->chgrp($ctools_path, d('@server_master')->web_group, TRUE) ->succeed('Change group ownership of settings.php to @path') ->fail('Could not change group ownership of settings.php to @path') ->status();
3. Enabling/disabling drush/provision extensions from hostmaster
Every drush extension can implement a hook_drush_load(), and check if a module is enabled in the hostmaster site. See provision.api.php for an example.
4. References
Extending Aegir and communicating with install profiles
This article has been republished from mig5's website. Original URL
Aegir is a pretty powerful tool that allows you to very quickly provision new Drupal sites out of the box and manage them throughout their lifespan through a variety of tasks.
Its ability to understand different install profiles and thus 'distributions' shoots the awesome factor through the roof once you can deploy instances of OpenAtrium, Pressflow, ManagingNews or your own custom distro in just a couple of clicks.
Inevitably, though, your requirements will grow more complex. Aegir is about automation, and install profiles that prompt users for information through a series of tasks, and fail without that information, aren't much help. You might resort to a bunch of hacks to get that data into your client's site. You might try and insert some custom stuff into the site's vhost only to whimper in dismay when Aegir does a routine Verify sanity check and obliterates your changes.
If you get to this point, the simplicity of Aegir will be a distant memory and you'll be shaking your fist at us developers for forgetting about your corner case. Curses!
Rest assured, despite the constantly shifting development in Aegir and its early alpha stages, we try to maintain an ability to hook into Aegir and extend it as easily as possible, and it will only continue to get easier, especially once our API stabilises and is published.
Today I'm going to show you:
- how to add a custom Hosting submodule to Aegir
- how to alter the site form to add custom fields of data
- how to send that data to the backend
- how to apply that data to site vhosts and inside your custom install profile and site database
There are a few files here, because we're creating a custom module and install profile, and a few other things. For the impatient, I've attached a tarball to this article called 'cheesy.tar.gz' that contains all the relevant components and a README.txt instructing where to put it all.
Those of you who follow me on Twitter know I like to perpetuate the semi-French stereotype and rant and rave about cheese. I spent some time trying to think of a non-cheesy example to use in this tutorial, before realising what a silly idea that was. You can never have enough cheese :)
Part one - Adding a cheesy module
We'll add a custom module to the Hosting frontend called 'Cheese'.
Hosting is the 'frontend' bit of the Aegir system that allows you to plug data in via a web interface and have that information, if necessary, communicated to the backend Drush/Provision system by way of 'tasks'.
Our new module is simple: we're going to invoke a hook_form_alter() to add a textfield to the site form. The module will add a table to the Aegir database that stores this value, linked to the site nid. Keeping it out of the hosting_site table means we can disable and uninstall this module later and clean up properly.
There's nothing new or unusual about a Hosting submodule than any other Drupal module. Make the directory /var/aegir/hostmaster-0.x/profiles/hostmaster/modules/hosting/cheese
Info file
Here's our hosting_cheese.info
name = Cheese
description = Cheesemongers need to define cheese in their Drupal sites.
package = Hosting
dependencies[] = hosting_site
core = 6.x
You can see we depend on hosting_site, because hosting_site is the module that provides the site form for installing sites!
Install file
Now let's add an .install file that defines our database table schema and how to install/uninstall it. Standard Drupal stuff.
<?php
/**
* Implementation of hook_schema().
*/
function hosting_cheese_schema() {
$schema['hosting_cheese'] = array(
'fields' => array(
'vid' => array(
'type' => 'int',
'not null' => TRUE,
'default' => 0,
),
'nid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'default' => 0,
),
'cheese' => array(
'type' => 'varchar',
'length' => 128,
'not null' => TRUE,
),
),
'indexes' => array(
'vid' => array('vid'),
'cheese' => array('cheese'),
),
);
return
$schema;
}
function
hosting_cheese_install() {
// Create tables.
drupal_install_schema('hosting_cheese');
}
function
hosting_cheese_uninstall() {
// Remove tables.
drupal_uninstall_schema('hosting_cheese');
}
?>
The module
Here's our simple hosting_cheese.module file.
It's pretty straightforward: our first function is a hook_form_alter()
of hosting_site_form()
. We are adding a single textfield to the form called 'Cheese'. Because it's my favourite, the default value is going to be 'camembert' if it hasn't already been set on the node (if we're updating a site rather than creating a new one).
The next few functions are invocations of common database hooks such as insert, update and delete. They will insert the 'cheese' value from our form into the hosting_cheese
table.
I call hook_nodeapi()
to do $stuff on those actions. There's validation to check that a cheese was entered. We invoke hook_load()
, which allows various modules and features to return their bits and pieces from various parts of the database as 'additions' to the core node attributes. With this we can call $node->cheese
when we might need to.
<?php
/**
* Implementation of hook_form_alter()
*/
function hosting_cheese_form_alter(&$form, $form_state, $form_id) {
if ($form_id == 'site_node_form') {
$form['cheese'] = array(
'#type' => 'textfield',
'#title' => t('Cheese'),
'#description' => t('What sort of cheese?'),
'#default_value' => $form['#node']->cheese ? $form['#node']->cheese : 'camembert',
'#required' => TRUE,
'#weight' => 0,
);
return $form;
}
}
/**
* Implementation of hook_insert()
*/
function hosting_cheese_insert($node) {
if ($node->cheese) {
db_query("INSERT INTO {hosting_cheese} (vid, nid, cheese) VALUES (%d, %d, '%s')", $node->vid, $node->nid, $node->cheese);
}
}
/**
* Implementation of hook_update()
*/
function hosting_cheese_update($node) {
db_query("UPDATE {hosting_cheese} SET cheese = '%s' WHERE nid = %d", $node->cheese, $node->nid);
}
/**
* Implementation of hook_delete()
*/
function hosting_cheese_delete($node) {
db_query("DELETE FROM {hosting_cheese} WHERE nid=%d", $node->nid);
}
/**
* Implementation of hook_delete_revision()
*/
function hosting_cheese_delete_revision($node) {
db_query("DELETE FROM {hosting_cheese} WHERE vid=%d", $node->vid);
}
/**
* Implementation of hook_nodeapi()
*/
function hosting_cheese_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL) {
if ($node->type == 'site') {
switch ($op) {
case 'insert':
hosting_cheese_insert($node);
break;
case 'update':
hosting_cheese_update($node);
break;
case 'delete' :
hosting_cheese_delete($node);
break;
case 'delete revision':
hosting_cheese_delete_revision($node);
break;
case 'validate' :
if (!$node->cheese) {
form_set_error('cheese', t('You must enter a cheese!'));
}
break;
case 'load':
$additions['cheese'] = db_result(db_query("SELECT cheese FROM {hosting_cheese} WHERE vid=%d", $node->vid));
return $additions;
break;
}
}
}
?>
At this point, you can go to /admin/build/modules
and enable the Cheese feature. The database table will get installed, and you can now go to Create Content > Site
and see our cheese field is present in the site form!
Part two - sending the cheese to the backend
Yes, it's very exciting that we have cheese in Aegir now. But don't provision a site yet! The fun hasn't even started.
In the same directory as the hosting_cheese module, create a new file called hosting_cheese.drush.inc
. This is a Drush file that is going to be responsible for catching when an Install or Verify task is being executed against a site, and to send the cheese value along for the ride to the backend.
<?php
/**
* Implementation of drush_hook_pre_hosting_task()
* Send the site's cheese attribute to the backend for processing.
*/
function drush_hosting_cheese_pre_hosting_task() {
$task =& drush_get_context('HOSTING_TASK');
if ($task->ref->type == 'site' && ($task->task_type == 'install' || $task->task_type == 'verify')) {
$task->options['cheese'] = $task->ref->cheese;
}
}
?>
In this code, $task->ref
represents the node object (the site).
Because of our additions in our implementation of hook_node_load(), the cheese value is fetched and available for us to use.
Part three - an install profile (just for fun)
I'm demonstrating all this to show you how to get data from the frontend (in the site form) to the backend. We've achieved that, but you probably want to actually *do* stuff with the data.
I'll demonstrate two things you can do with this data. They are just examples - Aegir isn't limited to this at all.
cheesy_sites.profile
First let's create a simple install profile called 'Cheesy sites'.
<?php
/**
* Return an array of the modules to be enabled when this profile is installed.
*
* @return
* An array of modules to be enabled.
*/
function cheesy_sites_profile_modules() {
return array(
/* core */ 'block', 'color', 'filter', 'help', 'menu', 'node', 'system', 'user', 'path',
/* other contrib */ 'install_profile_api', 'token', 'pathauto', 'views',
);
}
/**
* Return a description of the profile for the initial installation screen.
*
* @return
* An array with keys 'name' and 'description' describing this profile.
*/
function cheesy_sites_profile_details() {
return array(
'name' => 'Cheesy site',
'description' => 'Select this profile to install a really cheesy site.'
);
}
function
cheesy_sites_profile_tasks(&$task, $url) {
// Install dependencies
install_include(cheesy_sites_profile_modules());
// Fetch and set the cheese attribute from Drush, sent originally
// from the site form in the Aegir frontend.
variable_set('cheese' , drush_get_option('cheese', 'camembert'));
}
?>
This is not meant as an exercise in learning how to write install profiles so I'll gloss over some of the details here. Basic version: we define our modules that the profile depends on (core + some contrib), we install those modules, and finally we do a basic variable_set
to store a 'cheese' variable in the site's database, fetching the value from the Drush/Provision backend (sent by the frontend per above), defaulting to 'camembert' if there was none.
Obviously in a normal install profile, this file would be bigger, as you'd be doing a bunch of other stuff such as setting up node types, setting up menus, blocks, users etc.
cheesy_sites.make
I provide a Drush makefile here to fetch the contrib for you. This is totally out of the scope of the tutorial, but it's just fun to use Drush Make and I encourage you to get used to it :)
core = 6.x
api = 2
; Contrib modules
projects[token][version] = "1.15"
projects[pathauto][version] = "1.5"
projects[views][version] = "2.11"
projects[install_profile_api][version] = "2.1"
Download Drupal into /var/aegir/drupal-6.19 and copy the cheesy_sites directory into the profiles directory of drupal-6.19, or use a Drush stub makefile to do it all.
If you downloaded core manually, you'll want to cd drupal-6.19; drush make profiles/cheesy_sites/cheesy_sites.make --no-core
. The contrib will be downloaded to /var/aegir/drupal-6.19/sites/all/modules
Add the platform
Add a new Platform in Aegir with the Publish Path being that which we just created: /var/aegir/drupal-6.19. A Verify task will be spawned, and Aegir will pick up all the modules and also the 'cheesy_sites' profile, which you can check by viewing the 'Installation profiles' section of the 'Packages' menu tab on the platform node, after the Verify task has completed.
You could also add this profile and its dependencies to an existing platform and just re-Verify the platform, and the new profile will be synced up and added to this platform's package registry.
Part four - make the backend knowledgeable about cheese
Now we've set up the frontend to get the cheese, set up a drush file to send the cheese to the backend, and we have an install profile that expects to find cheese and set a variable about it in a database when installing a site.
We've left out one crucial stage, and that is to actually prepare the backend to tell it that cheese is coming!
cheese.drush.inc
Telling the backend about the incoming cheese is very similar to what we did earlier about capturing the cheese info on specific tasks and do $stuff with it.
To tell Drush and Provision (Aegir's Drush extension) about the cheese, create a file in ~aegir/.drush called cheese.drush.inc
.
We'll invoke a Drush hook that sets the value for the backend on installing a site.
<?php
/**
* Implementation of hook_pre_provision_install()
*/
function drush_cheese_pre_provision_install($url = NULL) {
drush_set_option('cheese', drush_get_option('cheese', 'camembert'), 'site');
}
?>
This saves the value into the site context, where it's usable by the install profile. This occurs in the pre hook, prior to when Aegir starts executing the tasks outlined by the install profile.
Part five - using Aegir hooks to inject settings into the site vhost
In Part four, we'll have accomplished what is needed to set the cheese value for the backend, which in turn means you can now create a site in the Aegir frontend, choose the 'Cheesy site' profile, and expect the install profile to pick that cheese setting up and store it in the new site's database as a variable.
But! While I'm here, let me show you another Aegir hook that allows you to safely inject extra configuration sent from the frontend into the site's Apache vhost config file, persistently, without it being overwritten next time the site is Verified.
We'll invoke an Aegir hook called provision_apache_vhost_config()
to insert a SetEnv into the vhost file, just for the hell of it.
Another example where you might want to do this is, say, to set a htpasswd password in the site form, and use the hook to inject mod_auth htpasswd protection for the site (perhaps a demonstration for another time!).
Add this to your cheese.drush.inc:
<?php
/*
* Implementation of hook_provision_apache_vhost_config()
*/
function cheese_provision_apache_vhost_config($uri, $data) {
return " SetEnv cheese " . drush_get_option('cheese', 'camembert');
}
?>
The return value is inserted into the vhost. You can return multiple lines by wrapping them in an array, or even more elegantly, simply return a path to a template .tpl.php file and do your customisations in the template!
Part six - check the results!
If you've done everything up to Part four and five prior to installing a site using the 'Cheesy site' install profile, do so now!
I'm going to create a site called 'cheesy.mig5-forge.net' and set my cheese type to be 'cheddar'.
Now let's check to see those settings in their end result:
The variable in the new site's database
Run this command: drush @yoursite.com vget cheese
You should see something like this:
Great! Now let's look at the Apache vhost file for this site, and we should see a 'SetEnv' parameter injected into the vhost.
Conclusion
I know these have been silly examples, but I hope you've taken some important lessons out of all this:
- How to extend the frontend of Aegir and hook into things like the site form and site node
- How to capture data on task 'events' like install and send it to the backend
- How to recognise that data in the backend and use it in install profiles or invoke our hooks to inject the data into configuration files
- How much I really like cheese
Cheesy tarball
As mentioned at the start of this article, you can download all these components I've been talking about here: cheesy.tar.gz. Remember to read the README.txt
Further reading
A great article by hadsie on g.d.o on sending data to an install profile via the Signup form covers similar information on capturing data sent to the backend in an install profile.
Steven Jones has published a HTTP mod_auth feature extension for injecting htpasswd password protection on sites over at https://github.com/computerminds/aegir_http_basic
E-Commerce Integration Roadmap
Related links:
- For installation and configuration instructions, see the Administrator Manual.
- There was some valuable discussion on use cases on groups.drupal.org.
Don't put feature requests in here. Submit them to the uc_hosting or hostmaster issue queues instead.
1. Drupal 6 / uc_hosting 1.0
Aiming for compatibility with:
- Hostmaster 1.0
- Ubercart 2.4
Structure:
- uc_hosting: This implements shared functions and classes, including the function to create a client based on an ubercart order and product.
- uc_hosting_products: This is where we put the ubercart function calls, attribute generation, etc... This module is intended to be used alone for the "my ubercart is on my aegir site" scenario.
1.1. Goals
1.1.1. Single site products
In.
1.1.2. Multi site packages
In.
1.1.3. Recurring payments
We need to test the module using uc_recurring.
1.2. Nice-to-haves
Given where we are at now, this stuff will most likely not make it in until an eventual 1.1 release. The exception being Hadsie's work on "try before you buy".
1.2.1. Purchasing wizard
Stills needs to be developed. The create my site now option is a start and is already in.
1.2.2. Make it unnecessary to log single-site purchasers into the aegir site
Needs some testing but should be doable. Maybe just needs a little documentation. I believe Hadsie's work is related to this.
1.2.3. Deferred payments
Ergonlogic is working on this.
2. Drupal 7 / uc_hosting 2.0
We are still unsure about whether to support commerce, ubercart, or both with this module. We will most likely wait until development on the D7 port of aegir starts before beginning this work so we will see then what the landscape looks like.
Aiming for compatibility with:
- Hostmaster 2.x
- Ubercart 3.x, or Commerce 1.x (maybe both?)
2.1. Goals
2.1.1. Remote storefronts
Hopefully, this will be supported by plans for new xmlrpc methods in the hostmaster signup module.
2.2. Nice-to-haves
2.2.1. Desjardins recurring payments
This is a seperate module being developed by Koumbit.
2.2.2. Importable demo products
This would be really great.
2.2.3. Simple procedure to deny user 1 to certain clients
Ergonlogic has published a module that should make this possible: http://drupal.org/project/hosting_profile_roles.
3. Wishlist:
4. Unanswered Questions
- Will the Drupal 7 version use Ubercart or Commerce? Both? Neither?
- How and when will people be billed? And what degree of control will uc_hosting offer over that?
- Aegir has clients & UC has clients. How are they related/distinct? How do we keep this clear to users/admins?
Creating product features using uc_hosting 1.x
This document assumes you want to implement a product feature using uc_hosting's database tables and exiting infrastructure. For a more complete example of an Ubercart 2.x product feature implementation, check out uc_file in the Ubercart core.
The Basics
Any module implementing a uc_hosting feature should include both uc_hosting and uc_hosting_products as dependencies. Include these lines in your .info file:
dependencies[] = uc_hosting
dependencies[] = uc_hosting_products
Declaring product features to Ubercart 2.x
The first step is to implement Ubercart's hook_product_feature. Here is an example implementation for the platform access feature in uc_hosting_products:
/**
* Implementation of hook_product_feature().
*/
function uc_hosting_products_product_feature () {
$features = array();
// Set the feature for the platforms
$features[] = array(
'id' => 'hosting_platform',
'title' => t('Access to a platform'),
'callback' => 'uc_hosting_products_platform_form',
'delete' => 'uc_hosting_products_feature_delete',
'settings' => 'uc_hosting_products_platform_settings',
);
return $features;
}
An implementation of hook_product_feature returns a numerical array of associative arrays. Feature arrays must include five keys:
- id will be used to identify your feature at the database level, in the uc_product_features table, and also in urls. Think of it as the type of your feature.
- title is the name of your feature as it will be displayed to site admins.
- callback will be used to create and modify instances of your feature attached to specific products. This callback should return a drupal form passed through the uc_product_feature_form function.
- delete is a simple function called on the removal of a feature instance from a product.
- settings is a callback that provides additional settings for the feature, and is not used presently in any uc_hosting product feature.
Of note is that, unlike Drupal's core hook_menu and hook_theme, hook_product_feature does not include a 'file' key. uc_hosting works around this by using include_once at the beginning of the hook_product_feature implementation.
If you wish to use uc_hosting's shared feature functions, make sure to include the file "products/inc/uc_hosting_products.feature_shared.inc" (path relative to the uc_hosting root) at the beginning of your implementation of hook_product_feature.
The callback form
This form is called everytime someone adds a product feature to a product. It allows you to set any options for your feature that should be delivered on a per-product basis. Lets take a look at this function for the platform access feature:
/**
* Callback to add the platform form on the product feature page
*/
function uc_hosting_products_platform_form ($form_state, $node, $feature) {
$form = array();
// Get default values and shared form elements for all uc_hosting features
$hosting_product = _uc_hosting_products_feature_fetch_product($feature);
if (empty($feature)) {
$feature = array('id' => 'hosting_platform');
}
_uc_hosting_products_feature_fetch_product attempts to retrieve existing values for this instance of a product feature from the database.
_uc_hosting_products_feature_shared_elements(&$form, $node, $feature, $hosting_product);
_uc_hosting_products_feature_shared_elements declares some form elements that either uc_hosting or Ubercart itself depend on to provide the product feature functionality.
// @see _hosting_get_platforms()
if (user_access('view locked platforms')) {
$platforms = _hosting_get_platforms();
}
else if (user_access('view platform')) {
$platforms = _hosting_get_enabled_platforms();
}
else {
$platforms = array();
drupal_set_message('You must have permission to access platforms to enable this feature.', 'warning');
}
$form['platform'] = array(
'#type' => 'select',
'#title' => t('Platform'),
'#description' => t('Select the platform to associate with this product.'),
'#default_value' => $hosting_product->value,
'#options' => $platforms,
'#required' => TRUE,
);
This code block generates a list of platforms to allow the admin to select which platform to bind to his product. This code is specific to the platform access implementation - this is where you should include your own form elements relevant to your feature.
$form['#validate'][] = 'uc_hosting_products_single_feature_validate';
For many implementations, uc_hosting_products_single_feature_validate ensures that a given uc_hosting product feature can only be enabled once on any given product. All of the exising uc_hosting_products features make use of this function.
return uc_product_feature_form ($form);
}
Finally, uc_product_feature_form adds some required form elements from Ubercart.
You also need to make sure to code a submit function for your callback, of the form callback_submit:
/**
* Save the platform feature settings.
*/
function uc_hosting_products_platform_form_submit($form, &$form_state) {
$hosting_product = array(
'pfid' => $form_state['values']['pfid'],
'model' => $form_state['values']['model'],
'type' => 'platform',
'value' => $form_state['values']['platform'],
);
$platform_node = node_load($hosting_product['value']);
$description = t('<strong>SKU:</strong> !sku<br />', array('!sku' => empty($hosting_product['model']) ? 'Any' : $hosting_product['model']));
$description .= t('<strong>Platform:</strong> !platform', array('!platform' => $platform_node->title));
$data = array(
'pfid' => $form_state['values']['pfid'],
'nid' => $form_state['values']['nid'],
'fid' => 'hosting_platform',
'description' => $description,
);
$form_state['redirect'] = uc_product_feature_save($data);
We manually save the product_feature to ubercart's database tables here. This is so that we can then retrieve the index of the entry in the uc_product_features table for later use in our own data.
// Insert or update uc_hosting_products table
if (empty($hosting_product['pfid'])) {
$hosting_product['pfid'] = db_last_insert_id('uc_product_features', 'pfid');
}
$existing_prod = db_fetch_object(db_query("SELECT hpid, data FROM {uc_hosting_products} WHERE pfid = %d LIMIT 1", $hosting_product['pfid']));
$key = NULL;
if ($existing_prod->hpid) {
$key = 'hpid';
$hosting_product['hpid'] = $existing_prod->hpid;
$hosting_product['data'] = unserialize($existing_prod->data);
$hosting_product['data']['platform'] = $hosting_product['value'];
}
// If necessary build a data array from scratch
else {
$hosting_product['data'] = array(
'platform' => $hosting_product['value'],
);
}
$hosting_product['data'] = serialize($hosting_product['data']);
drupal_write_record('uc_hosting_products', $hosting_product, $key);
}
Note that the uc_hosting_products table has both an integer column value, and a longtext column data for storing serialized information about the feature.
Taking action in Aegir based on product features
uc_hosting assumes that most purchases made via Ubercart are intended to be expressed as changes to an Aegir client node. So it provides some code to help you create new clients or modify exiting ones when an order containing your feature is made. Any other actions needed to make your feature work you will need to implement yourself. This will involve implementing the Ubercart hook, hook_order and your own callback function.
Lets start by taking a look at uc_hosting_products own implementation of hook_order:
/**
* Implementation of hook_order
*
* Actions t$form_state['values']['create_later']o take on order changes involving an aegir product
*
* @param $op string
* Provided by ubercart on invocation
* @param &$arg1
* Different data depending on the op
* @param $arg2
* Different data depending on the op
*/
function uc_hosting_products_order ($op, &$arg1, $arg2) {
switch ($op) {
case 'update':
if ($arg2 == 'completed') {
foreach ($arg1->products as $product) {
if (_uc_hosting_products_has_feature ($product)) {
// Make the changes necessary to the hosting client
$client = uc_hosting_update_client($arg1, $product, 'uc_hosting_products_client_update');
}
}
}
break;
default:
break;
}
}
_uc_hosting_products_has_feature is defined in uc_hosting_products.module, and provides some simple logic to detect uc_hosting product features. Rather than use this function, you will most likely want to define your own conditions.
uc_hosting_update_client does the work of matching up an incoming ubercart order to an existing Aegir client, if possible. If not possible, it creates a new client node. It then calls the function provided as a third argument, in this case uc_hosting_products_client_update, and passes to that function the affected client node, as well as the product and order objects from Ubercart.
This callback function is the place to take actions in Aegir or pass commands. Take a look at uc_hosting_products for a fully fleshed out example.
Class auto-loading
Provision has been refactored in 6.x-2.x to use class auto-loading. This simplifies use of classes, as it obviates the need to explicitly include the files in which the class and it's parent classes are defined. In addition, it standardizes the placement of class definitions within a hierarchical file-system structure and will make name-spacing simpler if/when we begin to adopt Symfony components.
While not strictly required in contributed modules, it is highly encouraged. If you maintain a contributed Provision extension, some re-factoring will be required to support class auto-loading.
First off, you'll need to add a registration function that will enable Provision to autoload your extensions classes. Here's an example from provision_civicrm:
/**
* Register our directory as a place to find Provision classes.
*
* This allows Provision to autoload our classes, so that we don't need to
* specifically include the files before we use the class.
*/
function civicrm_provision_register_autoload() {
static $loaded = FALSE;
if (!$loaded) {
$loaded = TRUE;
$list = drush_commandfile_list();
$provision_dir = dirname($list['provision']);
include_once($provision_dir . '/provision.inc');
include_once($provision_dir . '/provision.service.inc');
provision_autoload_register_prefix('Provision_', dirname(__FILE__));
}
}
Next, your extension will need to call your new function in a hook_drush_init(), so that it is run as early as possible during a Drush bootstrap.
/**
* Implements hook_drush_init().
*/
function provision_civicrm_drush_init() {
// Register our service classes for autoloading.
civicrm_provision_register_autoload();
...
Finally, you can move your class definitions into the proper file-system structure. In provision_civicrm, we defined a new (stub) service to allow saving data from the front-end into a site context (entity/alias). So, we created a file at Provision/Service/civicrm.php that included the class definition:
/
* The civicrm service base class.
*/
class Provision_Service_civicrm extends Provision_Service {
public $service = 'civicrm';
/
* Add the civicrm_version property to the site context.
*/
static function subscribe_site($context) {
$context->setProperty('civicrm_version');
}
}
Customizing Platform Templates
Non-standard Aegir installs may require adjustments to the default configurations for a platform that are defined by Aegir during the creation of a platform. These templates can be found in /home/aegir/.drush/provision.
Editing the actual templates prevents Aegir from overwriting your customizations when the verify task is run on an live platform.
Below are examples of customizations made to non-standard Aegir installs
Running an Aegir platform on a unix socket for php-fpm on Nginx
First let's find out where provision has the default config templates for our platfom.
[root@host]cd /home/aegir/.drush; grep -nir "fastcgi_pass" --exclude=*.svn* *
provision/http/nginx/nginx_advanced_include.conf:227: fastcgi_pass 127.0.0.1:9000; ### php-fpm listening on port 9000
provision/http/nginx/nginx_advanced_include.conf:360: fastcgi_pass 127.0.0.1:9000; ### php-fpm listening on port 9000
provision/http/nginx/nginx_simple_include.conf:213: fastcgi_pass 127.0.0.1:9000; ### php-fpm listening on port 9000
provision/http/nginx/nginx_simple_include.conf:346: fastcgi_pass 127.0.0.1:9000; ### php-fpm listening on port 9000
provision/http/nginx/nginx_advanced_include.conf:227: # fastcgi_pass 127.0.0.1:9000; ### php-fpm listening on port 9000
provision/http/nginx/nginx_advanced_include.conf:228: fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
provision/http/nginx/nginx_advanced_include.conf:361: #fastcgi_pass 127.0.0.1:9000; ### php-fpm listening on port 9000
provision/http/nginx/nginx_advanced_include.conf:362: fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
provision/http/nginx/nginx_simple_include.conf:213: #fastcgi_pass 127.0.0.1:9000; ### php-fpm listening on port 9000
provision/http/nginx/nginx_simple_include.conf:214: fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
provision/http/nginx/nginx_simple_include.conf:347:# fastcgi_pass 127.0.0.1:9000; ### php-fpm lis#tening on port 9000
provision/http/nginx/nginx_simple_include.conf:348: fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
Remember to do the same to your platform config files in /home/aegir/config/includes. Next time you verify your platform your customizations will remain after verification.
Passing values into the site drushrc.php file during migrations
It's possible rely to on manipulating the drush context for certain tasks and using hook_post_command to make use of that data. In this case, I wanted to save values to the site's drushrc.php file.
Add the following code to hook_post_provision_verify
:
<?php
if (d()->type == 'site') {
$val = drush_get_option('my_custom_option');
drush_set_option('my_custom_option', $val, 'site');
}
?>
This makes sure that my_custom_option
perpetuates through these potentially destructive operations. Since it is provision-verify that writes the drushrc.php file for the site, drush options set in the site context during hook_post_provision_verify
will be saved.
The final step is to make sure the verify hook is run with the appropriate options at the end of my migrate task, so add this to hook_post_provision_migrate
.
<?php
if (d()->type == 'site') {
$options = array('my_custom_option' => drush_get_option('my_custom_option'));
$target = drush_get_option('target_name');
provision_backend_invoke($target, 'provision-verify', array(), $options);
}
?>
I got this idea from looking at the drush_provision_drupal_provision_migrate function
in platform/migrate.provision.inc
.
I used this approach to create a patch for provision_civicrm
This was originally documented in the provision issue queue.
Integrating Aegir
This section of the documentation deals with integrating Aegir components with other services (as opposed to extending or overriding direct Aegir functionality itself).
Examples might include using Aegir's database to authenticate users based on their presence as 'clients' in the Aegir system, e.g authenticating FTP access to sites for clients that have those sites managed in Aegir.
Integrating Aegir with OpenSSH
Adam has written an article that explains how to hook up OpenSSH and PAM on your Linux server, so that you can authenticate your clients via the Aegir database and have them SFTP/SSH into your server and be chrooted (locked) into just seeing their sites (or a platform) with a restricted command(s) shell.
Integrating Aegir with PureFTP
Cafuego has written an excellent article that explains how to hook up PureFTPd and PAM on your Linux server, so that you can authenticate your clients via the Aegir database and have them FTP into your server and be chrooted (locked) into just seeing their sites.
Continuous Integration
Testing server URL: http://ci.aegirproject.org
Recently, mig5 put together a testing server based on Jenkins.
So what? In doing so, he's produced a tool that will allow us to spin up a temporary virtual server, drag in our codebase from Drush Make, other codebases as platforms, and do test installs of sites on it. This should become a routine start-to-finish test of our applications, features, modules, etc. All at the click of a button (or less, by scheduling it!)
Currently, we have a build project that:
- Provisions a new VPS at Rackspace Cloud
- Installs Aegir 6.x-1.x branch
- Builds a Drupal 6, Drupal 7 and OpenAtrium platform using Drush Make
- Installs a site (with respective install profiles) on all three platforms
If anything fails here, an alert will come through to #aegir on Freenode. This will help us to test the Aegir install (especially to ensure our compatibility with Drush and Drush Make releases), and to identify any regressions in our own project.
Every 15 minutes, Jenkins will check if there's a new commit to hostmaster and provision, and if there is, fire up a new install and test a few things to find regressions.
We also have a build project for the most recent official release of Aegir (@TODO: test .deb installation method of Aegir for this), as well as a build that installs an earlier Aegir and then upgrades to the latest 6.x-1.x to test the upgrade path.
Also, steven has created a sandbox to host the code that runs the tests on the jenkins server.
Despamming guide
We have seen numerous waves of spam on this site. At first we setup mollom, in 2011 but sometimes it is not enough.
Current practices
The very first step is to identify the user account and disable it (not delete it, we need it to find the content to remove).
Views Bulk Operations views used for despamming:
Note that this will flood your screen with drupal_set_message
for every page that is deleted, which may make the site difficult to use for a while. To top that, the final page will list the name of every page that was removed, just to make sure you had enough spam for the day.
You can also use the regular comment view to delete content and report it to mollom if you are patient enough.
You can also delete the account if you are motivated and want to grind that specific axe. Look at the regular user listing for the latest user accounts created.
Other ideas
Presentation & Training Materials
This section of the handbook is for various presentation and training materials about Aegir. This includes videos, slides, speaker notes, high-resolution images, etc.
Presentation videos
In this section of the handbook, we collate various footage of Aegir presentations and demonstrations for convenient access.
Contribute!
Feel free to add child pages and embed footage of Aegir that you may have found or demonstrated at a DrupalCamp, conference or meetup, or just to provide help to the community. Please try and follow the standards outlined in the other video pages, whereby the title includes the location, and the date and the name of the presenter is listed.
Note on deprecated videos
Please note that Aegir is a constantly evolving project under heavy development. Many of these videos contain footage and information that is deprecated (out of date or obsolete) compared to current versions of Aegir.
These older videos are maintained here for historical purposes, but should not be relied upon to successfully use or install Aegir. You should always refer to the Install documentation and the INSTALL.txt of the latest release of Aegir at the time you are reading this, as it is likely that much has changed since these videos were published.
Need help?
Always ask for help if you are not sure on how to install or use Aegir! These videos are a supplementary guide or demonstration of specific features only.
2010 DrupalCamp Montreal: Antoine Beaupré -- Aegir: One Drupal to Rule Them All
The Aegir Project's lead maintainer, Antoine Beaupré (anarcat), presents an overview of Aegir at DrupalCamp Montreal.
Date: October 23rd, 2010
URL: http://hd.drupalcampmontreal.com/watch/antoine-beaupr%C3%A9-aegir-one-dr...
2010 DrupalCon Copenhagen - Aegir - One Drupal to Rule them All!
Date: August 2010
Presenter(s): Adrian Rossouw, Antoine Beaupré
http://cph2010.drupal.org/sessions/aegir-one-drupal-rule-them-all
2010 DrupalCamp LA - How Aegir Will Save Your Sanity
Date: August 2010
Presenter(s): John Fiala
2010 DrupalCon San Francisco - Aegir - One Drupal to Rule them All!
Date: May 2010
Presenter(s): Miguel Jacq, Antoine Beaupré
http://sf2010.drupal.org/conference/sessions/aegir-hosting-system-one-dr...
This session will illustrate the use of the Aegir hosting system to simplify the life of developers and administrators, by automating a lot of the common tasks involved in deploying sites.
Aegir is a distributed provisioning system for Drupal that allows you to manage thousands of sites across as many concurrent instances of Drupal on as many servers as you need. It's built on Drupal itself, so that your user interface to the system becomes a 'meta-drupal' site, with nodes representing all of your hosted sites and all of the components of your hosting environment.
If you have built your own custom niche application using Drupal, and need to be able to roll out new sites quickly and efficiently, as well ask keeping them up to date, this presentation is definitely for you.
If you have more than a handful of sites, and feel your hair stand on end every time a security release is announced, this presentation is for you.
Even if you only have a few sites, and would like to be able to spend less time on the tedious manual tasks associated with running a Drupal site over it's entire lifetime, you will still find something of use in this presentation.
If you've looked at Aegir in the past, and would like to get a sense of the progress made by the project (two major releases since Drupalcon DC), this presentation is a good way to get a grasp on where we are heading.
For more information on Aegir, check out the Aegir user group and keep up to date with the Aegir posts on developmentseed.org.
In Norse mythology, Aegir was the god of the oceans and if a Drupal site is a drop of water, Aegir is the deity of large bodies of water.
2009 DrupalCon Paris - Automate your maintenance troubles away with the Aegir hosting system!
Date: September, 2009
Presenter(s): Adrian Rossouw, Antoine Beaupré
2009 DrupalCon Paris - Aegir: Build Once, Deploy often: Real life use-cases
Date: September 2009
Presenter(s): Roel De Meester
2009 DrupalCon DC - Deploying and Maintaining Drupal Sites Using the Aegir Hosting System
Date: March 2009
Presenter(s): Adrian Rossouw
2008 DrupalCon Szeged - Deploying and maintaining Drupal sites using Aegir hosting system
Date: August 2008
Presenter(s): Adrian Rossouw
2009 Drupal Melbourne Meet Oct - Introduction to the Aegir Hosting System
Date: October 2009
Presenter(s): Miguel Jacq
2009 DrupalCamp Montréal - Aegir - One Drupal to Rule them All!
Date: October 2009
Presenter(s): Antoine Beaupré
2011 DrupalCamp Spain -- Aegir: One Drupal to Rule Them All (Spanish)
Juan Pablo Novillo (juampy) present an overview of Aegir at DrupalCamp Spain.
Date: 2nd of October, 2011
URL: http://2011.drupalcamp.es/sesiones/aegir-un-drupal-para-gobernarlos-todos
Video: http://www.archive.org/details/AegirUnDrupalParaGobernarlosATodos
2011 DrupalCon London -- Aegir Based Business Models
An Aegir Project documentation & community manager, as well as contrib developer, Christopher Gervais (ergonlogic), presents an overview of business models supported by Aegir, as well as a demo implementing a simple Software-as-a-Service platform.
Date: August 23rd, 2011
URL: http://london2011.drupal.org/conference/sessions/aegir-based-business-mo...
2011 DrupalCon London -- Aegir: One Drupal to Rule Them All
Two of the Aegir Project's maintainers, Antoine Beaupré (anarcat) and Steven Jones (darthsteven), present an overview of Aegir at DrupalCon London.
Date: August 24th, 2011
URL: http://london2011.drupal.org/conference/sessions/aegir-one-drupal-rule-t...
Alternative URL: https://www.youtube.com/watch?v=Uub18h_9PMY
2014 Drupal Melbourne Meetup March 11 -- Web Stacks for local development
Brian Gilbert (@BrianGilbert_) of Realityloop regarding web stacks for local development, Aegir wins.
Date: March 11th, 2014
2014 DrupalCon Amsterdam: Spidersnet Infinity: Creating a Site Factory with Drupal & Aegir
Date: October 1rd, 2014
URL: https://amsterdam2014.drupal.org/session/spidersnet-infinity-creating-si...
2009 Drupal Vancouver April meet - Automated Deployment with the Ægir Hosting System
Date: April 2009
Presenter(s): Robin Puga
Screencasts
This section of the handbook is reserved for screencasts that demonstrate various parts of Aegir in a commented video.
Contribute!
Feel free to add child pages and embed footage of Aegir that you may have found or created yourself. Please try and follow the standards outlined in the other video pages.
Note on deprecated videos
Please note that Aegir is a constantly evolving project under heavy development. Many of these videos contain footage and information that is deprecated (out of date or obsolete) compared to current versions of Aegir.
These older videos are maintained here for historical purposes, but should not be relied upon to successfully use or install Aegir. You should always refer to the Install documentation and the INSTALL.txt of the latest release of Aegir at the time you are reading this, as it is likely that much has changed since these videos were published.
Need help?
Always ask for help if you are not sure on how to install or use Aegir! These videos are a supplementary guide or demonstration of specific features only.
Oct 2011 - Zero-Touch Drupal Deployment with Jenkins, Aegir, Git, Fabric and Drush
How to automatically deploy changes from dev, to stage, to production, without ever having to even login to Aegir.
Originally published at mig5.net.
Nov 2010 - Installing 0.4-beta1
Date: 23 November 2010
Presenter(s): mig5
http://greenbeedigital.com.au/aegir_videos/installing_aegir_0_4_beta_1.mp4 (110MB)
Nov 2010 - Upgrading Aegir
Date: 7 November 2010
Presenter(s): mig5
http://greenbeedigital.com.au/aegir_videos/upgrading_aegir.mp4 (98MB)
Drupaldojo - Introduction to the Aegir Hosting System
Date: May 2010
Presenter(s): Miguel Jacq
Building Drupal applications with Drush Make, provisioning sites with Aegir
Date: November 22, 2009
Presenter(s): mig5
http://greenbeedigital.com.au/aegir_videos/building_drupal_apps_drush_ma... (88MB)
Deprecated: Aegir from Scratch - Installing 0.2 RC1
Date: May 20, 2009
Presenter(s): Adrian Rossouw
Warning: this screencast is out of date, but may contain useful information.
We recommend you install the latest release of Aegir at the time you are reading this.
Deprecated: Applying Drupal Security updates with Aegir
Date: September 16, 2009
Presenter(s): Eric Gundersen, Jeff Miccolis
Warning: this screencast is out of date, but may contain useful information.
We recommend you install the latest release of Aegir at the time you are reading this.
Deprecated: Installing Aegir 0.4 alpha5
Date: Feb 1, 2010
Presenter(s): mig5
Warning: this screencast is out of date, but may contain useful information.
We recommend you install the latest release of Aegir at the time you are reading this.
http://greenbeedigital.com.au/aegir_videos/installing_aegir_0_1_alpha5.mp4 (199MB)
Aegir Images
The Aegir logo and related images have been given to the Aegir community by Development Seed in the early stages of development. The full source is available in the hostmaster repository, in the images source directory of the eldir theme.
Feel free to upload other high-resolution images of Aegir-related materials.
Article archive
This area contains archives of articles related to Aegir.
Aegir at DrupalCon San Francisco
This article first appeared on mig5's blog on 27th April, 2010
The Aegir project has seen vigorous development leading up to, during and after DrupalCon San Francisco April 2010.
Unfortunately due to the volcano ash from Iceland, lead developer and Aegir author Adrian Rossouw got marooned in Frankfurt and was unable to join the rest of the team at Moscone.
When Aegir 0.3 was released and I joined the development team, Adrian mentioned in the release notes that the 'bus factor' threatening to affect the progress of the project was reduced thanks to have extra hands on deck. In a way, although it was a devastating disappointment that he was unable to make it to DrupalCon, the opportunity arose to test what we renamed to be 'the volcano factor' and I think we did well under the circumstances.
On the initial Code Sprint Sunday, Antoine and I sat down and ripped apart the Hostmaster install profile wizard. After a good 5 or 6 hours, Aegir became capable of being installed from the commandline directly, with no need to progress through the traditional browser-based wizard. Significantly, this install profile task wizard has always been one of the most common areas that new users struggle with when installing Aegir, as steps can be missed or misunderstood very easily.
Now with the new install script (which is currently compatible with our dev-services branch in git), one need only install the usual dependencies (such as Apache, MySQL etc) and then execute the install.sh script, which takes care of the rest. Seconds later, a Login URL is given to you and you can immediately access the Aegir frontend!
Amongst the sessions on Monday, we also participated in an informal Aegir BOF session that was organised by Brian Wood, much to our delight. In this session, we gave a brief overview on where Aegir was headed, but focused on hearing from you guys, our user base, on how you're using Aegir and what issues / benefits you've experienced that we could take back with us.
That evening after the BOF, we organised an informal Aegir dinner to carry on further discussions. I had a very nice vegetarian Pad Thai, and later we made our way to the Lullabot and official DrupalCon parties :)
As we approached Wednesday, the Aegir team knuckled down and actually got around to preparing an agenda for our session that got voted in (thanks!). For myself this also meant preparing a local dev demo environment to demonstrate the latest features in Aegir, as well as testing all the demonstrations we'd exhibit and make sure they actually worked :)
Somewhere along the line, both Antoine and myself were interviewed by Lullabot for their famous Drupal Voices podcast series, where we discussed Aegir, best practices for dev/stage/live workflows, Drush and Drush Make. Expect to hear these interviews in the coming months as they trickle into the Drupalsphere.
For good measure, Adrian discovered why our once-working Drupal7 site provisioning was broken (as we expected, it was due to changes upstream in d7 core) and fixed the issue. At that point we released 0.4 alpha7, just in time prior to our session :)
On Wednesday 21st April at 16:15pm, we presented the "Aegir - One Drupal to Rule Them All" session, covering:
1) a basic introduction into the project
2) features/functions of aegir
3) what's changed since Paris
4) development/security practices
5) what's coming
6) some Q/A time
We felt the session was a success and it was great to see so many people show up and ask a lot of good questions.
You can watch the full session here or download/watch on archive.org
After the conference, we made it to the Thursday code sprint at San Francisco State University, and nailed a few more bugs / features! What was also discovered was that one can upgrade a vanilla Drupal 6.16 site to the latest Drupal 7 using our Migrate feature in Aegir, now that an upgrade path exists and drush updatedb succeeds. Wow!
DrupalCon SF 2010 is over but progress continues. All the bulk of our hard work at the moment is within the "dev-services" branches of the provision and hostmaster git repositories on git.aegirproject.org, where our new object-oriented refactoring and introduction of the abstracted 'server' concept and service-based API is paving the way for true multi-server support.. not far away now!
You're welcome to checkout these branches and start playing with it and fix some issues, or even just to test out the new command-line-only installation process. However, things are changing all the time, so it's volatile and there may be things that are very broken. Definitely don't use the dev-services branches in production!
Thanks to all who made it to DrupalCon or watched from a distance, attended or are listening/watching our session, came and introduced themselves and/or had a beer with us at some point, or are simply just using Aegir and loving it :)
I've some more exciting things to discuss in Aegirworld, but those will have to wait :)
Hope to see you at the next DrupalCon!
Edit: I forgot to also give thanks to two other groups:
1) the organising committee for DrupalCon itself - awesome job!
2) and also the guys at Koumbit for sending along a great team including Antoine, and equipping them with great Aegir 'One Drupal to Rule them All' stickers :)
An introduction to the Aegir Hosting system
This article first appeared on mig5's blog on 13th October, 2009
Tonight at 18:30pm GMT+11 I'll be giving a talk/presentation on 'An Introduction to the Aegir Hosting system' at the regular monthly Melbourne Drupal meet @ the Emspace offices.
My notes during the last week kind of took on a life of their own, and so in anticipation of the talk, I thought I'd publish them here, despite the somewhat casual tone, for those who can't make it and want a somewhat (perhaps overly, for an Introduction!) detailed overview of the Aegir system and what it does.
I believe the session will also be recorded, so we should see the video pop up on blip.tv or somewhere in days to come. Update 20/10/09: The video is available here http://blip.tv/file/2741212
A variation of these notes may end up being part of the documentation on g.d.o too.
Update: Here's a PDF of these notes http://ln-s.net/4Nbk
---------------------------------------------------------------------------
An introduction to the Aegir Hosting system
Contents
- What is Ægir?
- Brief history of Ægir
- Dissecting gods: the anatomy of Ægir
- Features and terminology
- What Aegir doesn't do
- Roadmap - where are we going
- Questions
- Links
What is Ægir?
Ægir is a set of Drupal components that when used together, help you manage other Drupal sites.
It does this by providing you with a simple Drupal based hosting frontend or control panel that allows you to perform actions against your network of Drupal sites. Some of these actions or 'tasks' include Installing new sites, Verifying the integrity of sites, backing up and restore, management of 'Platforms' (which are copies of Core or distributions such as OpenAtrium), and migration of your sites between these platforms. We'll cover the relationship between sites and platforms in a moment.
To Ægir, a site is just a node like any other node in a Drupal site. Depending on the node type, be it Site or Platform etc, determines what tasks like these can be performed.
Taken from the page on groups.drupal.org: In Norse mythology, Ægir was the god of the oceans and if Drupal is a drop of water, Ægir is the deity of large bodies of water.
Brief history of Ægir
Ægir used to be called Hostmaster, or Hostmaster2, and was authored by Adrian Rossouw who remains the lead developer of the project, back when he worked at Bryght. I believe Bryght pioneered the idea of a system like this to manage large deployments of sites for their clients, and Hostmaster was the result of that work and was subject to much growth and enhancements over a period of 4 years before it ever became 'Aegir' as it is now. Adrian's also known for other Drupal accomplishments such as the Forms API, the PHPTemplate theme engine, and the install profile system.
Since then, Bryght got acquired by Raincity Studios, and Adrian's now working for Development Seed.. thus a large influence on the development/direction of Ægir is naturally coming from that camp as they provide Adrian with the time and the means to work on improving the product.
Joining Ægir as part of the core development team is Antoine Beaupré, aka 'anarcat', who's from the Koumbit worker's collective in Montreal and been working on this for a long time too, and myself.
For this introductory talk, I'm going to focus on the anatomy of Ægir and how the components fit together and with Drupal. Then I'll provide a brief overview of the main features of Ægir and what you'd use it for.
After the talk, to avoid taking up too much time, I'll be happy to run through some more in depth, under-the-hood areas of Ægir such as the Provision backend, the database schema etc, and answer any of the more technical questions you might have.
Dissecting gods: the anatomy of Ægir
Ægir is not a 'module' nor yet a 'distribution' for Drupal, but currently a collection of 'components' as I like to call them, that work together to do the whole job.
In order to best understand how it all works, we can define Ægir as made up of two parts: the frontend, and the backend.
Frontend
The frontend is actually a Drupal site that makes use of a module called 'Hosting', which does all the job of the frontend: basically it provides the mechanisms to add / remove nodes like any other Drupal site, allowing you to point and click on tasks like Install, Backup and so forth. It also provides a nice UI to manage your sites in general, and to review tasks that have completed, or in progress, or failed for whatever reason. Most screenshots you've probably seen of Ægir, are using the Eldir theme developed by Young Hahn at Development Seed, which is made specifically for Ægir, though it is optional and Ægir will function without it.
Backend
The backend is made up of a 'drush module' or extension called Provision, which does pretty much its namesake: it works with Drush very heavily to handle all of the tasks such as installing new drupal sites, importing existing ones, installing platforms, verifying everything, instigating backups and restores, and so on. So it's actually doing the heavy lifting here, actually executing the tasks you assign to the system.
Provision is invoked through the use of drush commands, and thus these commands can also generally be executed from the command line to perform the functions that make Ægir so important. In fact it's very much by design that all the magic happens in the backend, because it allows us to have one Ægir frontend that can manage multiple backends, or in otherwords, multiple webservers and database servers. Though these are experimental features at the moment.
How Hosting and Provision work together
The glue that binds what Hosting lets you kick off tasks for, and how Provision learns of those tasks and executes them, is by way of adding the tasks to a Task queue in the database, which is viewable in the frontend.
On your server, we then have a dedicated 'aegir' user, whose crontab has an entry to regularly look and 'dispatch' any queues containing tasks that are pending, which are then forked off into drush provision commands in the background.
So the idea is that all this happens behind the scenes for you as the manager of the aegir system: the frontend exists for you to queue up all these sorts of actions, and let Ægir worry about actually doing them in the backend.
Along with Provision, Hosting, and Eldir, there is also another component which is an Install Profile called 'Hostmaster'. The Hostmaster Install profile is, like all install profiles, really only executed once, when you are setting up Ægir for the first time. And it does the work of getting information from you, or telling you to set things up on a system level (like directories, permissions etc), asking what Features you'd like to enable, and kicking off the initial verification of the system, setting up the crontab entry etc.
The home directory of Aegir
The backend Provision module
The Hostmaster profile and the Hosting frontend module contained therein
The Features and terminology of Ægir
So before I start running a few examples of what Ægir can do, I thought I'd bore you some more with the terminology and features that Ægir has, as there's often some confusion over what some of the terms mean.
Some features in Ægir are enabled by default, some cannot be turned off, some are optional and can be enabled at any time, and some are experimental and may have unexpected results in production environments.
In keeping with strong Drupal tradition, and out of pure sense, everything is a node in Ægir. This includes Ægir's information on your web server, database server, platforms, sites, right down to tasks, and information about modules, themes, and install profiles.
First of all, Ægir's very much designed to work within Drupal's multisite structure. Often one of the biggest hurdles users experience when trying to learn Ægir, can be traced back to the fact that few of them are actually making use of Drupal's multisite design, and instead tend to use one core per site. While Ægir will work that way just fine, its designed to make use of the proper multisite logic introduced way back in I think Drupal 4.5 or maybe d5 - in fact, it's what makes it so powerful and useful - it handles multisite for you, and it handles it properly.
For whatever reason, multisite is often not used, and so users are often confused about what constitutes a 'Platform' in Ægir, so we'll cover that now.
Basically the first thing you do after installing Ægir, is go and set up what we call a 'Platform', which is the code base within which you'll create a Site to sit on.
'Platform' translates directly to a copy of Drupal core, like Drupal 6.14 or 5.20, but it isn't limited to Drupal core. It could very well be a 'distribution' or custom Drupal core like OpenAtrium, Pressflow, OpenPublish, and so on. So it's important to recognise that a Platform is actually very simple to understand conceptually once you learn to exchange the term for a 'Drupal core' or thereabouts, and I suspect it's often expected to be more complicated than it really is.
So just per a standard multisite system in Drupal, sites are installed into the sites/ subdirectory of that Platform.
So a Platform and a Site are two examples of what are simply node types in the database, just like any regular Drupal site. This is what makes Ægir so easy, because to create a new Site, you can literally node/add/site or Create Content > Site and fill out the required fields, submit the form, and Ægir does the rest.
So you start to form a picture in your mind of a pyramid structure: In Drupal multisite, a Core or a Platform manages many sites. Ægir is a level above that, in that it can manage many Platforms. So Ægir becomes very powerful, because through inheritance of managing many Cores or Platforms, it exponentially can manage many sites, be they Drupal 5, Drupal 6, Pressflow, or Openatrium sites. And so on.
When you add a new Platform or site, you can see that these items are immediately dropped into the task queue which you can see in this block in the sidebar. As the cron comes around once per minute, it'll pick these tasks up and dispatch them to the backend to be executed.
Depending on the return from Drush and Provision as they execute these tasks, the status of this task in the queue will change from Queued to either Failed or Success. In both cases, the full drush log output is given in the task node.
On a failed task, you can review the problems and if you feel you've fixed the issue, re-schedule the task to re-enter the queue and be executed again.
Drush and rolling back changes
Because Drush has a rich API that provides validate_, pre_, post_ and rollback hooks, we can safely revert any changes that were made if a task fails, through a rollback, or perform other functionality before or after the fact. This means that although Ægir is taking a lot of the heavy lifting out of your hands with this stuff, it's also very careful with your data. It doesn't want to destroy your sites, it wants to nurture them, and it'll undo or refuse to perform any actions it thinks might be unsafe.
In the case of installing a site, support is written for Drupal 5 and Drupal 6 platforms. There is Drupal 7 support in there that once worked, but the nature of Drupal 7's shifting meant that support for d7 fell out of Drush, and thus it currently doesn't work in Ægir, though I think a Drupal 7 platform *will* verify ok.
Installing sites works because Provision basically knows how to create a mySQL database and credentials automatically for the site and apply these settings, along with locale, timezone etc, and any other settings you normally enter manually during a Drupal install, automatically into the installer. It doesn't matter what installer it is: it can be either the standard Drupal default installer, or a custom install profile you've written, or other install profiles such as the atrium_installer profile that makes OpenAtrium what it is.
Install also goes through and generates the Apache vhost configuration file for the site and restarts Apache, so you don't need to do any of this.
On successful installation of a site, the onetime login reset link will be e-mailed to the 'Client' (we'll go through Clients later), and the login url will also be logged to the task output for quick reference. It's normally the task output that I'll go to to retrieve the login url. So immediately you can login to your brand new drupal site, and you only clicked a couple of buttons.
Ægir supports site 'aliases', which translate directly to the ServerAlias parameter in Apache vhost files. One can add URL aliases that will serve a site called by another name. A classic case is installing a site with a URL of 'www.example.com' with an alias of 'example.com' so that example.com also serves the site (providing a RR is set in DNS of course).
Redirection is also supported as an optional feature to Aliases, allowing permanent redirects of the aliases to the main site URL by way of mod_rewrite. All this is configurable in the site node form, though Site Aliasing is an optional feature that must be turned on first.
It is possible to import existing sites that you're hosting on a server, into your new Ægir system.
Since most people seem to be using one core per site and not using multisite, it's just a matter of moving the copy of core into /var/aegir/ , renaming sites/greenbeedigital.com.au to sites/$yoursite, and then adding a Platform in the Ægir frontend. Upon a successful Verify of the new Platform, any detected sites living in the sites/ folder will have an Import task spawned for them, and Ægir will hence 'learn' of these sites and their packages.
Obviously it is not ideal to have one platform per site, and the whole point of Ægir is to support multiple sites on one Platform. So once this site is imported, you will likely want to Migrate the site to another platform, which we'll cover in a moment.
Verify, and Packages
Along with Install, other tasks exist in Ægir. One that is often run is Verify, which just checks that the filesystem of the site or platform looks sane, permissions all correctly set, confirms the aegir superuser can create new databases, and so on. it also refreshes the database with knowledge of what modules, themes or profiles (altogether referred to as Packages in Ægir) exist on a platform of site. It keeps a running state of what packages are installed, so that if you try to Migrate a site, it knows whether the target platform can support these modules or not.
If you edit a site or platform's node - for instance if you are adding a URL alias for an existing site - on node_save, the item will be re-added to the task queue as a Verify task. Such functions serve to handle such cases like an alias addition, where the site's vhost needs to be edited and Apache restarted for the change to take effect.
If a site is installed and enabled, one can Disable the site, which effectively puts a redirect on the apache vhost for that site to redirect to a page saying the site's been disabled. It also goes and backs up the site.
If a site is Disabled, it can then also be re-enabled, which essentially unsets the redirection in the vhost so that the site starts working again.
After a site is in a Disabled state, you can Delete a site, which will go through and (again) take a backup first, then goes and removes the actual site data from the server, removes the knowledge of the site from the aegir database, drops the site's own database, and removes the vhost config. As mentioned earlier, if any of these steps go downhill, the procedure will be reversed as best as it can.
I mentioned Migrate earlier, and this is really in my opinion, the biggest feature of Ægir and what makes it so worth using. Migrate essentially is another task that tells Provision to *move* the site from one platform to another. The most obvious case of where you'd want to do this is when a new Drupal core release comes out, like recently with 6.14.
All I had to do is download Drupal 6.14 to the server, add it as a platform in Ægir, and then use Migrate to move my Drupal 6.13 sites to Drupal 6.14.
In the process of this, it runs the Drupal updates and hence performs the hook_updates, schema changes etc that one would normally do by hand in an upgrade.
What makes quite significant is that it's by design that you can take a Drupal 5 site, and use Migrate to upgrade it to Drupal 6. It's built right into Ægir to handle that upgrade automatically for you.
Migrate basically takes a backup of the site, and then it uses a special 'deploy' command to 'deploy' this instance of the site to the target platform. It then removes the site from the existing platform.
I mentioned earlier the concept of a 'Package' in Ægir, which is any module or theme or install profile that Ægir finds on a platform or a site. With its knowledge of the package, its schema version and what 'instance' in which this package is being used, it can analyse a site that you want to migrate, compare the packages in use with what can be used on the target platform, and present a report on whether the site can be migrated or not.
In other words, it will intelligently calculate that you cannot migrate a Drupal 6 site to a Drupal 5 site, because the schema versions of Drupal 5 packages are incompatible with those installed on the Drupal 6 site. However obviously, the reverse ought to work.
And yes, it backs up and will rollback if unsuccessful. :)
So I've mentioned Backup a few times, so I should cover it: essentially the Backup task can be executed against a Site. It does a mysqldump of the site into a file in the site's directory, and then it tars up the whole site's directory and drops the tarball into a backups directory that lives outside the document root of any platform with a timestamp.
Using the Restore task, you can take any of these backups, and with a single click, restore your entire site, codebase and database all together, to a previous state from one of the tarballs.
The 'Deploy' task I mentioned earlier isn't shown in the frontend. It's a backend command that is used in Migrate and Clone tasks. Clone is a new feature in our early 0.4 alpha releases that does what you probably are already thinking it does: it generates a snapshot of a site (actually using the Backup task above, we reuse as much code as possible) and uses Deploy to deploy that copy of a site with a new URL to any platform, be it the current platform or a separate platform.
So you can use Clone to clone entire copies of your site to a new URL, which could be useful for testing, or if you have a sort of 'template' site with common modules and themes, structures or common nodes etc. You could consider it a poor man's install profile.
To date, you could only Migrate a site on a per-site basis. We've recently introduced Batch migrate, which means you can migrate *every* site that currently exists on a platform, and move them all to a new platform in one hit. Again, especially useful when you have a lot of sites on a Drupal core that has suddenly become vulnerable with a new security release, and you need to move them all across to the new core as soon as you can.
We know of Ægir being used to deploy and manage more than 2100 sites on one system, actually by another Australian, Dave Hall, who's up in Bendigo, and goes by the alias skwashd on Drupal.org and IRC. So features like Batch Migrate are especially significant.
Since Ægir already has a task queue scheduler, it makes sense that it should be able to queue up enabled sites and run cron regularly on the sites themselves. So a Feature exists to do exactly that. Both the regularity of the Cron and the Task queues are configurable.
Another Feature and node type is that of the Client. A Client in Ægir is not a specific user in the typical Drupal sense, but more so a conceptual wrapper node that also translates directly to a role type. One then can create users and assign them to a 'Client' just like a role. Such users have the ability to login to the Ægir site and manage their own sites. In this way you can expose Aegir functionality to your clients and let them create their own sites or manage them (to a point).
Another role that Ægir sets up is the Ægir Account Manager: this is design as a non-technical role, likely a member of your own staff, who has the ability to set up and manage Clients in Ægir, but cannot perform any technical tasks or install sites etc.
A lot of work has been done in recent releases to implement multiple webserver Port support, along with SSL support which is not quite complete.
Currently one can create only one site URL on one port, but cannot create the same site URL on another port. There are cases where this will be required (such as port 80 and 443 for sites that need SSL as well), which is something I'm working at the moment in my development.
What Aegir doesn't do
- Aegir knows about your modules and their release schema etc, in order to handle migrations and that sort of thing. But it isn't a plugin manager, and you can't disable/enable modules of sites within Ægir.
- To the dismay of at least 50% of users who install it, Ægir is not in fact a build management tool, it's a site management tool. But one can build Platforms using drush_make, which can pull from drupal.org repos as well as your own git/svn/cvs repository, and migrate your sites onto new 'release' builds. I'll spend some time on this after the talk for those who are interested.
- It doesn't provision non-Drupal sites. But it may support static sites, Joomla? in the future..
Roadmap - where are we going?
- Lots of UI refactoring will be done before 1.0 - modal dialog, more UI feedback to the user
- Refactoring 'Servers' from being separate node types i.e webserver, dbserver etc, to a single 'Server' type that acts as a container for pluggable Services (http, mysql, postgresql, dns)
- Develop a File service, one reason is for dealing with moving backups around.. 'spoke' model or 'mesh' model (see roadmap)
- Just in this week gone, in the alpha2 release, we leverage 'Drush make', allowing Hostmaster to provision itself and migrate itself to new platforms (0.4 alpha2 and onwards likely)
- Third-party application hook-ins, i.e DNS, LDAP, Mail, Jabber.. with the idea of a control panel done right
- Stronger quota management and the potential for e-commerce hook-ins, selling sites as a product
- Move to PDO to support PostgreSQL, SQLite etc
- The 1.0 release will see a 'frozen' Hostmaster API
What else?
Questions?
Links
- http://groups.drupal.org/aegir
- http://drupal.org/project/hostmaster
- http://drupal.org/project/hosting
- http://drupal.org/project/provision
- http://drupal.org/project/eldir
- http://drupal.org/project/drush
- http://drupal.org/project/drush_make
- Aegir issue queues
- Migrate your Drupal sites to a new Drupal core security release with Ægir
- Automate your site maintenance troubles away with the Aegir hosting system - presentation by Adrian and Antoine at DrupalCon Paris
- 0.4 Roadmap
Drupal deployments & workflows with version control, drush_make, and Aegir
This article first appeared on mig5's blog on 28th October, 2009.
It's the million dollar question. And it's rarely been answered because it's so darn hard to do so.
Today I'd like to tell you how I do it with my favourite deployment weapons: Aegir, Version control, and Drush Make.
This'll be a rather low-level article designed for power-users. For a more high-level overview of the benefits described, check out the recent blog post by Development Seed.
The Problem
While many or most of us are using version control to keep track of our code and so on, a problem still persists, and it is twofold:
1) A lot of important data gets stored in the database in many database-driven content management systems, such as Drupal
2) The transition of updates through version control to the site can be a royal PITA.
Recently 1) has pretty much been conquered thanks to Features, which provides the ability to export the 'mechanics' of a Drupal site from the database into module-like code, which can then be version-controlled.
Adrian Rossouw covered how Features can help solve the classic Dev / Stage / Production workflow problem in a great blog post. Features is one of those big guns we've all been waiting for. I mean, google 'development live workflow', and it's this post that is number 1.
With the rising popularity of the Aegir hosting system, and its ability to fire off sites like a proverbial machine gun, the other question is becoming even more important.
Because Aegir is this fantastic system for managing your sites, right? Of course, I would say that. But how can you keep a site under version control when Aegir's capable of making and manipulating sites almost on-the-fly with the click of a button?
The Solution
Enter: Drush Make
The answer has come in the form of Drush Make, which provides an extension to Drush, the command line tool I often refer to as 'apt-get' for Drupal.
What Drush Make does, is through the implementation of flat files (similar in appearance to .info files from modules), provide a mechanism to fetch data from practically any source you tell it to, and do $stuff.
This can include downloading Drupal core, as well as a bunch of contrib modules from drupal.org . However, it also supports the ability to fetch:
- code from CVS repos, including tag-specific
- ditto for svn
- ditto for git, including checking out a particular branch after cloning the repo
- grabs .tar.gz stuff, wget-style, useful for libraries
- can fetch and apply patches
Sure, you say, that's great. It's basically doing batch drush dl
's with some extra features/protocols thrown in. But how does this help with version control? Not only that, but how is that going to help keep this stuff under control while managing the site in Aegir?
Certainly for me, much of the issue has been that Aegir is inherently 'on the fly' with its actions. It has no concept of version control. But eventually I realised I had to stop expecting Aegir to handle this aspect, and realise that I could work with it by bringing version-controlled data to it for processing.
Or, put another way, I realised the very act of moving data around, does not need to be a version-control procedure in itself.
One must start to think like Aegir: and that is that the Platform and its application is the important data, that needs to be controlled. The site is just an instance of the application, and so long as your application is controlled, your site can be manipulated as an instance of it.
Adrian mentioned it in his podcast interview with Lullabot on October 15th:
It helps to think of install profiles as applications, and sites as instances of those applications.
If you can grasp that, you're on the home stretch already.
The Platform becomes the Platform Build.
One of the concepts I struggle with the most when training users on Aegir, is that of the 'Platform'. It turns out that many developers haven't been using Drupal's multisite design even prior to Aegir, despite the fact that the design's been around forever. So the idea of having multiple sites inside the /sites/ folder of Drupal core - in other words, sharing the core codebase between sites - is somehow (often, but not always) already enough of a paradigm shift.
In Aegir, one creates Platforms, which are just code bases, or a copy of Drupal core (or Open Atrium, etc). Only then can we create and manage Sites, which live on Platforms.
The analogy is directly like this:
/var/aegir/drupal-6.14 << this the core, the Platform
/var/aegir/drupal-6.14/sites/www.mig5.net << this is the site, 'on' or 'inside' the Platform
What makes this hard is that some users are still getting used to this idea, even though Aegir isn't doing anything different here. It's just exposing that multisite design of Drupal, by making it easier.
If you can grasp the idea of multisite, you'll find it easier to understand how Aegir works with your sites, and how a workflow can be developed to handle build management.
I mentioned Drush Make, and it's ability to grab the Drupal core.
And I mentioned shifting one's mindset to think of Install profiles as just 'Profiles', or 'Applications', and sites as instances of those Applications.
With this in mind, a bigger picture starts to form regarding the potential of Drush Make, since it can, in one fell swoop,
- download the core,
- fetch the install profile from your git repo
- fetch some contrib modules
- fetch some custom themes and modules from your other repos
- maybe apply a patch against a module
- fetch some extra third-party libraries, i.e jquery.ui, or geshi
No, really. Look at this (entry-level) example of mig5_net.build:
core = 6.x
projects[] = drupal
projects[mig5_net][type] = "profile"
projects[mig5_net][download][type] = "git"
projects[mig5_net][download][url] = "git@git.mig5.net:/drupal/profiles/mig5_net"
projects[mig5_net][download][branch] = "build_2009101601"
Executable from the command line like this:
php /var/aegir/drush/drush.php make mig5_net.build /var/aegir/drupal-6.14_build_2009101601
My install profile or 'application' is the git repo that I'm cloning. Note the branch reference: after Drush_make 'git clone's it, it'll check out a specific branch, or 'build'. This is how this release is different from past and future builds: it's a specific release I'm building for.
Suddenly with this one execution of a single flat file, you've built a copy of Drupal core, and dropped in the profile or 'application' into its /profiles/ directory. That's a platform in Aegir's eyes, and not just any platform: but what I call a 'Platform build'. It's a specific release of a your application, right from the top of core, down to the last css file of your site's theme.
Hang on, .css file? Where's the contrib and all that? You only cloned the install profile from git
Yes, in the makefile above, I'm only grabbing the core and my install profile application. Here's the secret: it turns out Drush Make has a recursive nature! I tuck away another makefile inside my install profile repo, called mig5_net.make, and this is the real gem.
core = 6.x
projects[] = admin_menu
projects[] = captcha
projects[] = geshifilter
projects[] = install_profile_api
projects[] = pathauto
projects[] = recaptcha
projects[] = tagadelic
projects[] = token
projects[] = twitter
projects[] = views
projects[singular][type] = "theme"
projects[singular][download][type] = "git"
projects[singular][download][url] = "git@git.mig5.net:/drupal/themes/singular"
projects[tao][location] = "http://code.developmentseed.org/fserver"
projects[mig5][type] = "module"
projects[mig5][download][type] = "git"
projects[mig5][download][url] = "git@git.mig5.net:/drupal/modules/mig5"
What we have here is a series of contrib modules from drupal.org. Also I have a slightly hacked version of singular (css stuff), so I don't use Development Seed's feature server repo to grab this: I keep my own copy in my own git server. However, I *do* use the Tao base theme from their feature server, because I know, like contrib, I don't make any changes to it.
Alternatively, if I was smarter, I could have grabbed Singular from DevSeed's feature server, and written a patch for the css change, and asked Drush Make to apply the patch instead. Here's a snippet of such a thing, taken from the Managing News makefile in its install profile, where the Context module is patched:
; Patched.
; Explicit versions specified to ensure patches apply cleanly.
projects[context][subdir] = "contrib"
projects[context][version] = "2.0-beta7"
projects[context][patch][] = "http://drupal.org/files/issues/606816-1_node_form_context.patch"
...just so you don't think I'm making this stuff up :) Yes, such features exist, and are in active use!
So, this make file gets downloaded when the install profile git repo gets cloned and the build-specific branch is checked out. While Drush Make grabs the profile from the first makefile (I call it the .build file), it scans the directory and finds this make file, and recursively executes it - thus grabbing all the site-specific components in the process.
Time to tell Aegir about the new Platform build
Suddenly what I have is the entire platform, complete with any updated components I made in the makefile of that specific branch in git. At this point, I just have to tell Aegir about it!
So I go to Create Content > Platform in Aegir, give it a name (usually suffixed with the build number I give it as above), and tell it the path to the new platform I made (/var/aegir/drupal-6.14_build_2009101601)
Aegir goes and checks the platform, learns about it, adds the package information it finds within it to its database, and if all's well, gives me the big green tick.
But where's the settings.php? The site files? This isn't actually the *site*!
Exactly. It's only the application that we've built, and it's only the application we store in version control and generate with makefiles. The site still exists on the old platform, and Aegir knows it does. What we have to do here is stop thinking of the /sites/www.mig5.net/settings.php etc as the *site*, but rather, the current running 'instance' of the application.
It sounds a bit weird, doesn't it? But you don't realise, if you're already using Aegir, you're already probably doing this already. Drupal 6.14 came out recently, and your sites were on 6.13. You built a new platform (6.14, by downloading it), told Aegir about it, and you used the Migrate task to upgrade all your 6.13 sites to 6.14 at the click of a button.
We're doing exactly the same thing here: you've got your new platform, with your updates. Now you just need to Migrate the site onto that new build. Your settings.php and files from /sites/www.mig5.net/ get carried across to the new build, and any component code/database updates will get applied in the process.
Suddenly, you've just exercised the perfect workflow:
- You kept the application, important stuff, in version control
- You managed to keep *less* of a) in version control because drush make can fetch the contrib you don't hack, from drupal.org and other sources
- You built the whole new release with just one command on the shell
- You managed to seamlessly upgrade your site onto the new release, applying all the updates, and
- you did that last one via Aegir, which means you were protected with the rollback functionality of Provision and Drush, meaning if it all went pear-shaped, you'd be back on the previous release as though nothing had happened.
This is how Aegir can cooperate with version control: all it needs from you is to get that Platform, or build, in place, so it can use it and take care of the release process for you.
And while it used to be difficult to build an entire distribution complete with all the nitty gritty components required, it's now never been easier to do this step in a matter of seconds, because Drush Make does this job for you.
This application, www.mig5.net, is literally two files in my git repository, plus the theme (I hacked on Young Hahn's Singular theme a bit). The two files are the install profile itself (which, of course, I only ever once: first time install of the site) and the mig5_net.make file, that does everything else.
Dealing with the dev > live process
This really packs a powerful punch when you're working on a development version of your site to make changes prior to sending live (and if you're not, you should be). The process above fits perfectly into this model. This is how one does it:
Need to make changes?
Dev
- Make a new build using drush_make to grab the latest bits of everything. Add that platform to Aegir. I might call this build 'drupal-6.14_build_dev_2009102601'
- Use the 'Clone' feature to clone your current Live site to the test build.
- Make your changes to the dev site (maybe you are changing the theme, or adding a module)
- Commit those changes back to the repo (the theme changes, maybe adding that new module as a ''project' in your drush_make file inside your install profile). Make a branch of the repo in preparation for a new live build.
Live
- Make a new build using drush_make to grab the latest bits of everything (this grabs the changes you just committed). Add that platform to Aegir. I might call this build 'drupal-6.14_build_live_2009102601'
- Use the 'Migrate' feature to migrate your live site to the new build, which picks up and applies any changes/upgrades
I shared some ideas with Adrian Simmons (@adrinux) who has taken the build/deployment logic I describe here and generated an absolutely fantastic workflow diagram showing the transition of data from dev to live, as well as the relationship between drush_make, version control, and Aegir. Here is the diagram - it does a hell of a better job explaining it all than me and my wordy ways :)
The status of Drush Make
Awesomeness.
No, you'd know by now that I'd say that, but that's actually the release notes for the 6.x-2.0-beta1 release, which Dmitri unleashed today, after an amazing night (well, my time) by Adrian Rossouw who obliterated a bunch of bugs after being given commit access. The Aegir project now depends on drush_make to build the frontend system and its relevant components, as of HEAD and the upcoming release 0.4-alpha3 (stay tuned for that very shortly). For this reason, we have a direct interest in the project and its future, and so we were lucky to work with Dmitri and squash a few issues very quickly - some of which were actually identified during the writing of this article :)
The recent changes to Drush Make include bringing it in line with changes in the also-recently-released Drush 2.1, adding support for checking out git branches after a git clone, and supporting absolute paths for destination of builds on the server.
Need to play with it some more to try it out? You can see examples of my 'build' makefiles in my git repo as well as my application install profile for mig5.net, which contains the 'bigger' drush_make file. Or check out the Managing News install profile and its make file - it's the first project other than Aegir to use a make file to build itself.
If you're new to Aegir, Drush and Drush make, I hope this article gives you the incentive to try it out and see why it rocks. If you're using Aegir and Drush already, I hope Drush Make makes more sense now, and that this gives you that 'ah hah!' moment if you've been scratching your head about how to handle build management in an Aegir environment.
(thanks to Adrian Simmons, Adrian Rossouw and Eric Gundersen for their contributions/ideas for this article, and most of all to Dmitri Gaskin for his mad skills.. thanks for changing the game with Drush Make!)
Test your Drupal distro: Building a continuous integration server with Aegir & Jenkins
This article first appeared on mig5's blog on 9th June, 2011.
A while ago I wrote that I had started to use Jenkins for various purposes such as server backups, providing better notification in event of failure, yadda yadda.
Over at the Aegir project I implemented Jenkins to give us a continuous integration platform that kicks into gear either on demand or whenever one of us pushes a commit to the Drupal git repos. It does the following:
- Provisions a new virtual server (at Rackspace Cloud) via the Libcloud API
- Installs Aegir on the server
- Builds a Drupal 6, Drupal 7 and OpenAtrium platform via Drush Make
- Installs a site on each respective platform
A number of other build jobs exist, such as testing upgrades of earlier Aegir releases to latest versions, but I won't go into that here.
What several people have asked me to elaborate on, is that the combination of Jenkins, Cloud-based virtualisation and Aegir can make a very useful testing tool if you're building some sort of Drupal solution like a distribution or some sort of turnkey Drupal appliance where you want to repeatedly test the installation process (especially with a custom install profile, for example). This article will describe how we built such a thing.
Disclaimers before we start
1. I'll assume you have a server set up with Jenkins installed already, as that's out of the scope of this article.
2. The code I'm giving you uses Libcloud API, but it is very much designed for the Rackspace Cloud driver implementation. I know for a fact, through building other tools, that other provider drivers such as Linode have differing requirements (Linode, for example, requires you to also specify a datacentre to build in). Please don't contact me complaining that it didn't work with Provider X, because I'm just showing you how we use it with Rackspace. Since there's a good chance you're smarter than me, you're welcome to take the code and abstract it properly :)
3. The code is far from perfect because I'm rather crap at such things. The error handling is extremely crude.
4. I am not going to explain every line of code in this article: this is targeted towards clever people who will read the code and understand what to tweak, what assumptions to add or remove to make it build their own project. Again, this is just what (currently) works for us.
5. I am assuming the Jenkins server is running on Debian Squeeze, where things like fabric and python-setuptools can simply be apt-get installed with no fuss.
Step 1) Install libcloud tools
Libcloud will provide the API necessary for provisioning a brand new virtual server at a provider such as Rackspace Cloud.
Run these commands as root / with sudo on your Jenkins server.
git clone git://github.com/apache/libcloud.git
You'll need python-setuptools on a Debian/Ubuntu system:
apt-get install python-setuptools
Install libcloud likeso:
cd libcloud
python setup.py install
Step 2) Install fabric
Fabric, a python library for communicating with servers over SSH, will run necessary commands to the new VPS to install Aegir and the rest, via SSH.
Run this command as root / with sudo on your Jenkins server.
apt-get install fabric
Step 3) Upload scripts to the Jenkins server and edit
Two files provide the magic that does the whole job of building a new VPS and installing Aegir and the rest. You want these two files on the Jenkins server for the Jenkins user to use.
The .ini file is for you to edit and enter your API keys, server specs and relevant details such as your email address, trusted IP for firewall whitelisting on the new VPS, and URL to fetch your makefiles from to build your project.
The other file with the .py extension is the actual python code that Jenkins will execute to build the whole thing.
You can clone my repo containing the two files from github:
git clone git://github.com/mig5/aegir_ci.git
The code assumes this directory containing those files are in the home directory of the Jenkins user, e.g /var/lib/jenkins/aegir_ci
It's at this point that you want to edit the aegir_6.x-1.1_install.py file and define what to build. In my example that I'm giving you, I'm fetching three Drush make stub or 'build' files from my github repo, named drupal6, drupal7 and openatrium .build respectively.
The code that does this is the function fab_install_platform()
.
Thus, this code is very much specific to building your project from a makefile, the fact that the makefile is fetchable over HTTP (per the ini config file, it fetches from my github repo) and also makes some assumptions on what install profile to use when installing the site.
The code that installs the site is the function fab_install_site()
, and takes two arguments: the platform per the above (e.g drupal6, drupal7 or openatrium), and the install profile name (default, standard or openatrium in this example). So this is where you want to change the install profile name and only install whatever platforms you want, based on the name of your makefile.
Step 4) Add a new Jenkins job
Add a New Job of type 'free-style software project'.
In the combobox 'Add Build step', choose 'Execute shell'.
In the text field that appears, enter this line:
python /var/lib/jenkins/aegir_ci/aegir_6.x-1.1_install.py
Choose any other settings you'd like in the job, such as notifications if a build fails, whether you want to automate this build on a scheduled basis (like a cron), and so on.
Step 5) Execute a Build
After you've saved the job, simply click 'Build Now' in the left sidebar and watch the Console Output to see the new server get provisioned, your Aegir installed and your distro or custom Drupal app built and a test site install processed.
Blowing it all away
My example code has a snippet at the bottom that completely destroys the VPS at the end. This is because our Aegir project builds on a scheduled basis, and that includes times where I'm asleep: I don't want these cruft VPS lingering around costing me money.
You may wish to prevent the auto-destruction so that you can login and take a look around, especially where there are issues. To do so, just comment out conn.destroy_node(node)
at the bottom of the script.
To login to Aegir, either find the one-time login URL in the Console Output of the job itself, or check your e-mail, where you should have received the link in the welcome message.
Upgrading Aegir itself to the latest version of Drupal
This article first appeared on mig5's blog on 26th May, 2011
Today, Drupal released a security/bugfix for Drupal 6 and 7, bringing the latest version of Drupal 6 to 6.22.
Aegir is well known for making site upgrades very easy, fast and safe by way of the Migrate task. But some users might be wondering how to make the Aegir 'frontend', which is a Drupal-6 site itself, run on the latest version of Drupal without having to reinstall Aegir entirely.
For instance, if you installed Aegir 1.0 or 1.1, your Aegir system is probably running on Drupal 6.20.
Fortunately, our Upgrade instructions for Aegir, while typically describing how to update your Aegir instance to a new release of Aegir specifically, can also be used as-is to perform an upgrade of your Aegir system to the latest version of Drupal only.
In other words, follow the Aegir upgrade instructions per that link above as though you were upgrading Aegir, but retain the same version number in the steps:
export OLD_AEGIR_DIR=/var/aegir/hostmaster-6.x-1.1
export AEGIR_VERSION=6.x-1.1
export AEGIR_DOMAIN=aegir.example.com
The next step in the upgrade would normally be:
cd $OLD_AEGIR_DIR
drush hostmaster-migrate $AEGIR_DOMAIN $HOME/hostmaster-$AEGIR_VERSION
The upgrade steps will actually build a new Drupal platform ($HOME/hostmaster-$AEGIR_VERSION) automatically using Drush Make.
However, you can't have two directories in /var/aegir called 'hostmaster-6.x-1.1' because Drush Make will abort on this.
To work around this, adjust the 'target platform' part of this command so that the new platform directory will have a unique name:
cd $OLD_AEGIR_DIR
drush hostmaster-migrate $AEGIR_DOMAIN $HOME/hostmaster-$AEGIR_VERSION-d6.22
The naming convention is arbitrary here: I am simply suffixing it with d6.22 so I understand why I did it if I forget later :)
So long as it's a unique new target platform, your Aegir instance will 'upgrade' itself by backing up and then moving the Aegir frontend onto the new Drupal 6.22 platform, then running through the rest of the upgrade/ update.php steps automatically.
Afterward, you can safely visit the old Hostmaster platform node (usually /hosting/c/platform_hostmaster) in your Aegir frontend and remove it by running the 'Delete' task. I recommend you don't simply rm -rf the old hostmaster platform manually, but let the Delete task take care of this, and other cleanup tasks, for you.
As always if you have questions or get stuck, let us know on the Community portal, on the mailing lists, the Support ticket queue or on IRC in #aegir on Freenode.
Good luck!
------
If you need a consultant to handle the tasks of installing, upgrading or troubleshooting your Aegir instance, consider reviewing our consultancy services. mig5 is a core developer of the opensource Aegir hosting system.
Slidedecks
Feel free to add a page for any presentations you give on Aegir, and make sure to attach your slides and speak notes.
Apple Keynote Theme for Aegir Presentations
I created this Keynote theme for my Aegir presentation (http://community.aegirproject.org/using-aegir-drupal-hosting-and-managem...).
I hope this may be helpful for others presenting on Aegir.
Aegir Ecology vs Drupal Gardens
-Aegir Ecology vs Drupal Gardens
Presentation given to Sacramento Drupal Users Group, June 2011, attended by Lullabot's quicksketch and others...
Comparing the Aegir Project ecosystem and the Drupal Gardens stack.
The Aegir Project is an open-sourced, libre and free Drupal hosting system that is already used in production to host thousands of Drupal sites in a managed networks.
Aegir allows the spawning of sites from popular company or community distro's such as Open Atrium, Open Public or COD (Conference Organizing Distribution).
There is also a growing, diverse ecosystem of Aegir contributed modules such as hosting payment integration to Ubercart or the automation of various hosting management tasks.
Aegir is composed of the Hostmaster installation profile, Drush and Provision and has an Aegir API for developers. Aegir is production ready and is the product of some of the best Drupal people in the community.
Drupal Gardens is an entry level hosting platform that is a product of Acquia and is built in Drupal 7. The hosting platform is proprietary and specific to the hosting provider.
Slides: Schedule info Time slot: 29 May 14:30 - 16:00 Room: 4 - BoF Room
http://sacdrupal.org/program/sessions/aegir-ecology-vs-drupal-gardens
DrupalCampNYC -- Aegir-based Business Models
Christopher Gervais & Antoine Beaupré recently presented Aegir at DrupalCamp NYC. The session was recorded, and we'll post a link to the recording as soon as it's available. We've attached a tarball of the session's S5.
ref.: http://drupalcampnyc.org/sessions/aegir-based-business-models
DrupalCon London 2011 -- Aegir-based Business Models
This is the presentation I gave at DrupalCon London.
It is in S5 format, for easy presentation in a browser. I have included the .rst sources, theme and makefile, so that it can be easily modified. For more info, look into rst2s5. Please share back any modifications you use.
NYCcamp 2015 - Aegir 3.0 deep dive
Aegir 3.0 deep dive with ergonlogic and helmo.
http://nyccamp.org/session/aegir-30-deep-dive-ergonlogic-and-helmo
OpenCamp in Dallas, TX - 2010
Attached are the slides I used for a presentation at http://OpenCa.mp/ last year. Feel free to use and edit them as needed. (Note: This was a keynote file, but has been exported to PPT, so some of the slides might not look right.)
Overview of Aegir and Dev/Stage/Production
I presented this at the Pacific Northwest Drupal Summit. Aegir received a warm reception which has nice to see. Feel free to use these slides in whatever way necessary. On images where credit is given, please make sure you give credit to the original authors. Thank you.
Pacific Northwest Drupal Summit
Key Points
- Overview of what Aegir is and what it can do.
- Introduction to key Aegir concepts
- Discussion of current successes & remaining issues with Aegir in our environment
- Our Dev/Stage/Production workflow
- Live demonstration
- ?'s and comments
Original Session Description
We have recently been developing a new hosting architecture utilizing the Aegir platform. In our quest to move towards a more professional development model of a development, staging/qa, and production server I have learned a lot about setting up Aegir and the subsystems necessary to successfully move a site from the dev server through the chain up to production. This session would cover at a high level installation and configuration of Aegir and leveraging their hub-spoke model to speed up movement of sites and to ensure a higher level of maintenance to all drupal installations.
Aegir proves immensely useful when porting sites from D5 to D6. We would also discuss our move towards git and how that does and does not dove-tail with Aegir. I would hope to showcase some of our pitfalls so that others can avoid them if they choose to use Aegir in the future.
This would most likely be a 30-35 minute presentation with the rest of the time being devoted to discussion.
Using Aegir for Drupal Hosting and Management - DrupalCampSC (2011-06-12)
Presentation description/abstract:
Using Aegir for Drupal Hosting and Management
This session will explore using Aegir to deploy, clone, upgrade, backup and restore Drupal web sites with live demonstrations on an actual Aegir server. Methods for creating new platforms by importing existing sites and using Drush Make files will also be presented. Using Aegir to deploy Drupal distributions such as Open Atrium, OpenPublish and COD will be demonstrated. In addition, this session will explain use cases for Aegir multi-server setups.
While the primary focus of the session will be using Aegir, system requirements and installation will be discussed briefly.
Presented at DrupalCampSC 2011 (http://drupalcampsc.com/)
See slides in PDF format or the link below:
http://public.iwork.com/document/?a=p53574453&d=Using_Aegir_for_Drupal_H...
Training Materials
Post or find training materials here for any local trainings on Aegir you may be interested in.
DrupalCon Training - Aegir Hosting System: Deep Dive
- 1. Intro to Aegir
- 2. Installation
- 3. Debugging
- 4. The Aegir interface
- 5. Creating Platforms
- 6. Sites
- 7. Aegir vs. the Drupal 7 Upgrade Path
- 8. Importing sites
- 9. Drush Make Advanced
- 10. Client management
- 11. Extending Aegir
- 12. Whiteboxing Aegir
- 13. Advanced Topics, Manual Uprades, and other Black Magic
1. Intro to Aegir
1.1. Community
- aegirproject.org
- #aegir
1.2. Terminology
- site
- platform
- task
1.3. Architecture
- Provision
- Hostmaster
2. Installation
2.1. The easy way
- using Debian packages
2.2. The hard way
- manual installs
2.3. Other operating systems
2.4. Remote database servers and clusters
3. Debugging
3.1. Checking your install
3.2. Manipulating the task queue and reading the task log
3.3. Checking crontab
4. The Aegir interface
4.1. Platforms & sites
4.2. Servers & services
4.3. Tasks & queues
5. Creating Platforms
5.1. Creating a Drupal x platform from scratch
5.2. Some handy drush commands and an explanation of node/add/platform
5.3. Creating a platform using a drush makefile
6. Sites
6.1. Creating a site on our new platform
6.2. Common Site tasks
- install
- clone
- backup
- restore
- migrate
6.3. Cron configuration via Aegir
6.4. Site aliases
7. Aegir vs. the Drupal 7 Upgrade Path
7.1. Create a 7.x platform from scratch
7.2. Migrate our site from 6.x to 7.x
8. Importing sites
8.1. Single sites
- provision-deploy
8.2. Complete platforms
- provision-verify
9. Drush Make Advanced
9.1. Patching and branching with git
9.2. Creating cascading makefiles
- using includes
- overrides
10. Client management
10.1. Clients & the quota api
11. Extending Aegir
11.1. Overview of Aegir contib modules
11.2. Scheduled backups & garbage collection
11.3. E-commerce integration
11.4. ACLs & SFTP
12. Whiteboxing Aegir
12.1. Theming with Eldir
12.2. Overriding the Aegir makefile
12.3. Overriding other files
- settings.php
- apache config files
13. Advanced Topics, Manual Uprades, and other Black Magic
13.1. Advanced Debugging
- running tasks by hand
- force-deleting
- ...
13.2. Manual upgrades
13.3. Aliases & contexts
13.4. Performance & security
13.5. Extending Provision & Hostmaster
Strategy
Aegir is about tools, not policy.
You'll hear the maintainers of Aegir mention this regularly, when discussing various technical topics. The point is to highlight a design goal of Aegir's, which is to enable various workflows, rather than impose any.
The same applies to the business models one adopts, and hopefully facilitates, with Aegir. This section of the handbook is intended to gather various documents about major initiatives within the Drupal community, and provide a place to discuss and analyze them from the perspective of the Aegir community.
As a first step, I'm proposing two such documents:
- Open Apps Standard Draft
- The goal for the Open App Standard is to define the purpose and high level components of an App infrastructure in Drupal, and to define the technical protocols required for the creation, use, and distribution of apps in the community and larger marketplace.
- Software-as-a-Service Business Model Specification
- The Software-as-a-Service (SaaS) model allows clients to purchase subscriptions to pre-built sites (whether pre-packaged profiles such as Open Atrium, or custom-built profiles), which are then provisioned for them by an Aegir system.
Open App Standard DRAFT
I personally find the formatting and layout of the current Open App Standard DRAFT page difficult to parse, and the lack of a Table of Contents makes navigation difficult.
Also, I was under the impression from the recent DrupalCon session, Taking Inventory of Drupal Products and App Stores, that that page was a wiki. I personally hope that that page is made into a wiki, and perhaps some of the changes I propose here be integrated into it.
Open App Standard Draft
The Open App Standard: Guidelines for the Creation, Use, and Distribution of Drupal Apps
- 1. Goals
- 2. Components of the Open App Standard
- 3. Participants in the Open App Standard
- 4. Apps: An Introduction
- 5. The App Philosophy
- 6. The Apps Ecosystem
- 7. The Apps Manifest API
- 8. Best Practices for Creating Apps
- 9. Roadmap for future work by the Open App Standard Working Group
1. Goals
The goal for the Open App Standard is to define the purpose and high level components of an App infrastructure in Drupal, and to define the technical protocols required for the creation, use, and distribution of apps in the community and larger marketplace.
Beyond creating a forum for best practices around apps, the high level goal of the project is to achieve interoperability of apps between sites so that the Drupal platform does not become fragmented as specific distributions become widely adopted. That is, an app which conforms to the open app standard will be capable of being installed onto any site which conforms to the same standard.
Further, an open app standard should serve to facilitate greater community contribution, by eliminating any confusion about apps and their purpose, by making them easy to build, and by creating opportunities for community members to both contribute to and benefit from the Drupal distributions and platforms in the marketplace today.
2. Components of the Open App Standard
The Open App Standard aims to provide definitions, technical specifications, and best practices around creating, using and distributing apps in Drupal. In particular, the standard aims to provide the following:
- Definition of an "app"
- Reasoning behind apps in Drupal
- Definition of components and key stakeholders in the Apps ecosystem
- Definition of the Apps API Architecture
- Best practices around how an App is created, contributed, and shared
- Best practices around how an App "marketplace" is constructed, owned, and operated
- A common understanding and agreement of what elements of the Apps ecosystem belong to the open source community
- Accepted practices for interacting with 3rd party service providers in building and selling Apps
- Framework for enforcing best practices to ensure Drupal community needs
3. Participants in the Open App Standard
The Open App Standard is a collaborative project initiated by the following organizations. Additional organizations interested in collaborating on this project are invited to join the discussion and participate in the development of the standard. The organizing participants are:
- SubHub http://www.subhub.com
- Phase2 Technology http://www.phase2technology.com
4. Apps: An Introduction
The purpose of this section is to introduce the concept of "apps" in Drupal.
4.1. What is an app?
Before we say what an App is it’s helpful to know who it’s for: An App is for the end user, typically a non-technical user of a Drupal site such as the site administrator. These are the users we are thinking about when we discuss Apps.
An App is an installable package which solves a concrete task specific use-case. The complexity of its installation should be hidden as much as possible keeping the process uniform and simple. The goal of the App concept is to make extending the functionality of a Drupal site with discrete functions in a polished, “user friendly” way its main goal.
An App is a piece of packaged functionality. Upon installation, an App delivers a polished solution for a specific need.
The goal of the “Apps” module is to reduce the number of steps required of users from discovery to usability for functionality for their Drupal sites. By creating a specific protocol for installing and configuring apps, we’ll streamline the end-user experience, making Drupal functionality more “pluggable” and inherently easier to use.
Technically, an App is a collection of new or existing Drupal functionality, including Modules, Features, and Themes along with some simple additional code that publishes the existence of the App to the rest of the App infrastructure.
What distinguishes an App from a Module or Feature is that the App includes all of the additional polish, end-user UI, configuration UI, sample content, and documentation that are needed to fully meet a specific need. Whereas Modules and Features are more developer-centric and provide building blocks for larger functionality, Apps are larger blocks of completed functionality.
Unlike a Distribution or Product, one App only solves one specific problem. A collection of Apps might be seen as a full Product, just as a collection of Modules and Features might be seen as a Distribution.
4.2. What is the purpose of an app?
The primary purpose of an app is to simplify and make consistent the discovery, installation, setup and use of a specific piece of functionality on a Drupal site.
Examples include a Mailchimp app, which synchronises site users with a MailChimp list, a Project Mapper app that provides geolocation around specific projects for an organization, and then displays them on a map, or an ImageSlideshow app, which displays your images in a slideshow.
5. The App Philosophy
5.1. Why do apps exist?
There are a couple of things about Apps that are important to understand. First, that apps exist to solve specific problems for specific users. And second, that apps are generally built for the benefit of Drupal consumers, as opposed to modules which are generally built for the benefit of developers or site builders. To explain further:
5.1.1. Apps exist to solve specific problems for specific users.
To be clear from the beginning: the Drupal framework for creating functionality (modules), for creating specific designs (themes), and for delivering combinations of functionality and design (distributions) is a solid framework, from which the concept of "apps" does not deviate, fork, or undermine.
Rather, apps adds another piece to this framework -- the ability to solve very specific problems, for very specific end-users in a highly user-friendly way, using Drupal. Apps do not seek to add blanketing functionality to every part of a site, or to provide multi-function solutions to complex problems. Apps resolve discrete, specific needs for very specific groups of users.
5.1.2. Apps are built for Drupal consumers
First, let's clarify the concept of "Drupal consumer." A Drupal consumer is a site builder or site administrator or site owner (such as a Blogger) who is using a Drupal site. Drupal consumers are often not comfortable working in the code of a web site and may not have the knowledge to do so. These consumers are not visitors to sites, but rather, the owners/administrators who make decisions about what content and functionality they need for their site, and require a solution to provide it.
Consumers & end users are a picky bunch. They say things like 'it's almost there, but I also need this ...' But in a distribution or a SaaS product, "almost" isn't good enough. "Almost" is what drives end users 'back' to find something else. Borrowing from the Pareto principle, Drupal itself solves 80% of these problems, but consumers of Drupal need that last 20%. Apps can solve this problem, by simplifying the process of adding functionality to one's Drupal web site.
5.2. What are the Components of an App?
At a minimum, an app should include the following components:
- specific functionality or the connection of the end user to a third party service that provides specific functionality
- the ability to install, uninstall, enable, & disable, cleanly settings or configurations for the end user to customize their use of the functionality descriptions, screenshots, and graphics that tell an end user what the app is, what it does, and how it is used
5.3. What Isn't an App?
The following are characteristics that preclude something from being an “app”:
- It is not a complete solution: if an app only provides raw materials or building blocks, of a solution, it is not a good app.
- It is trying to solve too many problems, or too big a problem: Apps that install functionality into many different places in a site, or that try to solve a large host of problems, will not be good apps. (e.g. A CRM app would likely be too large a problem to solve)
5.4. Can any module or theme be an App?
Apps are specific and not broad, in nature – they are designed to solve one problem, solve it well, and solve it completely. Generally, an app will both keep track of data in some way (as a content type, etc) as well as use it or display it in a specific way (on a map, etc). So, an app like “Ideation,” created and contributed by Development Seed, is a great example of an app that solves one problem, solves it well, and solves it completely. Ideation provides feedback functionality for organizations. It solves it well, with a beautiful interface and simple-to-configure set-up, and it solves it completely, giving users the ability to not only suggest ideas, but also give feedback on that idea and rate the idea; and giving administrators the ability to accept or remove ideas from rotation.
Many excellent modules would not be considered an app, but perform vital functionality as modules. For instance, modules like CTools or Panels add enormous functionality to a Drupal web site, but perform their function very broadly, and therefore do not fulfill the “specific functionality” guideline for apps.
5.5. What is the difference between apps and “Features?”
Apps need not be Features, but a really good start for an App is to simply create a Feature. You need a good use case and functionality that can be isolated, self-contained and individually exportable and Features support that. They can contain Content Types, Views, Permission settings, Panels pages, Context/Spaces/Strongarm/Boxes settings, etc. Here are some good resources for learning how to build a good Feature.
An App may actually consist of more than one Feature to encourage Features to focus on smaller blocks of general-purpose functionality.
5.6. How do Apps work with the rest of Drupal?
Apps are designed to work within the Drupal framework, not outside of it. The following diagram outlines how Apps work in concert with the Drupal framework, and how they relate to Drupal Core, modules, Features, distributions, and SaaS platforms.
5.7. Who can create an app?
App development is open to any member of the community. Apps can be developed by organizations or individuals who want to make their functionality simple to install, configure, and use with Drupal or with a specific distribution of Drupal such as OpenPublic. Organizations who want to integrate their third-party applications should also consider creating an app. Apps that link users to other applications such as analytics, data storage, or social media aggregation can provide new customers to companies seeking integration with open sources CMSs like Drupal distributions.
6. The Apps Ecosystem
The purpose of this section is to define the components of the "ecosystem" around Drupal apps, and to show the relationship between that ecosystem and the larger Drupal ecosystem.
6.1. General Concepts
- App
- a discrete piece of functionality that is focused around managing the UI interactions and data to perform a particular task. e.g. adding a mailchimp signup form block to a page’s region. The process of ‘installing’ an app should be a simple click operation for the end user and the app framework would take care of any underlying dependencies required.
- App Market
- An app market is the user interface that allows a Drupal site owner to extend the functionality of a site with apps located on a App Serve and provide payment processing and checkout functionality as needed.
6.2. Technical Components
- Distributions
- a collection of Drupal code combined with an installation profile to accomplish specific functionality for a user group
- Apps Manifest Specification
- A technical specification that describes which apps are available, what their dependencies are, and associated information such as descriptions, logos, and screenshots.
- Apps Manifest API
- the bi-directional interface between the App Server and the App Client which mostly consists of requests and deliveries of data that adheres to the Apps Manifest Specification.
- App Server
- An app server is responsible for publishing a catalogue of its apps, synchronising versions with d.o. and other appstores, allow searching over the catalogue, packaging and delivering the apps and providing a payment infrastructure. It is thought appstore servers will be focused on serving a particular domain problem, e.g. Open Public, SubHub, etc. The App Server is a provider of data that adheres to the Apps Manifest Specification.
- App Client
- An app client would allow the user to browse/discover a remote collection of apps, prompt for any necessary technical and monetary requirements and install them on the chosen site. A process that should convey a simple, robust and secure UX. It is required that the client abstract as much technical information away as possible making the process of using and extending a Drupal installation as simple as possible. It’s likely this would be a module. It is a consumer of data that adheres to the Apps Manifest Specification. Technically, one client could browse multiple “app markets” pointing to different servers, if desired.
- Apps module
- the open source Drupal module that allows for the installation and use of apps in a Drupal distribution. This is an implementation of an App Client as defined above.
6.3. Definition of Roles Involved in Apps
- App owner
- the person or organization who owns or maintains the code for the app.
- 3rd party service provider
- a company who provides a service that they would like to make available to a platform's app consumers through an app. (e.g. if MailChimp wants to sell its services to users of the SubHub platform through an app, MailChimp is the "3rd party service provider")
- Platform owner
- the person or organization who owns or maintains a distribution or SaaS platform which utilizes apps or an app market. This owner defines which App Client gets used, and which stores it provides. (e.g. SubHub is the "platform owner" of the SubHub SaaS offering; Phase2 Technology is the "platform owner" of the OpenPublic distribution)
- App market owner
- the person or organization who hosts an app server, curates a collection of apps for that app market, and makes the app market target specific platforms. Often, a platform owner will also be an app market owner. However, another organization could host an app market and implement it in an instance of an open source Drupal distribution, and in that case, this organization would be the “app market owner,” but not the “platform owner.”
- App consumers
- app consumers are the site administrators or site builders who make decisions about which functionality to include on their sites.
- Direct app consumers
- site administrators or builders who are making decisions for their own sites.
- Indirect app consumers
- site builders who are making decisions for their clients' sites. (e.g. a development shop using OpenPublic to implement sites for public sector clients chooses apps appropriate for each client. This development shop is an "indirect app consumer")
7. The Apps Manifest API
The purpose of this section is to describe the RESTful API between the server and the client
7.1. Apps RESTful API Requests
[DRAFT NOTE: This documentation is forthcoming]
7.2. Apps RESTful API Response
[DRAFT NOTE: This documentation is forthcoming]
7.3. Apps Manifest Specification
7.3.1. General Attributes
- Key
- Value
- Name
- The common name description of of your App. Examples: Project Mapper, Ideation, Section Fronts, etc.
- Description
- A more descriptive explanation of the App. This can be many paragraphs if needed. This should be plain text.
- Machine_name
- The machine name of the main module. This is most often the name of the folder and the .module file
- Version
- The version number of the App to be displayed in the App Console and Detail page
- Downloadable
- The key in the downloadables array for the main module
- Author
- The name of the App author. Plain text name of person/company/organization.
- Author_url
- A URL to the author's homepage/website.
- Screenshots
- This is an array of of image URLs. Multiple screenshots will make slidshow.
- Logo
- URL to the image logo. This is expected to be an icon/logo of 120 x 120px
- Demo content module
- The machine name of the module that contains the Demo Content. This module should be included within the the main App module, but it can be provided as a separate downloadable too. When possible this module should make use of the Default Content and Features modules
- Demo content description
- A description of what kinds of content and usability the demo content module will provide. This give the user an idea of what to expect when demo content is enabled.
- Dependencies
- These contain the module dependencies for the App. This is a normalized list of all modules keyed by module machine name with a value that represents the module & version. The value string are keys in the downlodables array. These modules will be downloaded into sites/all/modules.
-
Example:
"dependencies": {
"openlayers_views": "openlayers 7.x-2.0-alpha1",
"openlayers_plus": "openlayers_plus 7.x-1.0-alpha2",
"feeds": "feeds 7.x-2.0-alpha3",
"mapbox" : "mapbox 7.x-2.0-alpha0",
"geofield" : "geofield 7.x-1.0-alpha1",
"job_scheduler" : "job_scheduler 7.x-2.0-alpha2"
} - Libraries
- These contain the 3rd party library dependencies for the App. This is a normalized list of all libraries keyed by library name with a value that represents the library & version. The value string are keys in the downlodables array. These libraries will be downloaded into sites/all/libraries.
-
Example:
"libraries" : {
"openlayers_slim" : "openlayers_slim 1.7"
} - Downloadables
- This contains the entire list of all things to be downloaded. The keys in this array represent the values from the dependencies and libraries. The value of these entries is the full URL to the downloadable resource.
-
Example:
"downloadables": {
"openpublic_projects 7.x-1.0-alpha3" : "http://featureserver.phase2technology.com/sites/community.aegirproject.org/files/fserver/openpublic_projects-7.x-1.0-alpha3.tgz",
"openlayers 7.x-2.0-alpha1" : "http://ftp.drupal.org/files/projects/openlayers-7.x-2.0-alpha1.tar.gz",
"openlayers_plus 7.x-1.0-alpha2" : "http://featureserver.phase2technology.com/sites/community.aegirproject.org/files/fserver/openlayers_plus-7.x-1.0-alpha2.tgz",
"mapbox 7.x-2.0-alpha0" : "http://ftp.drupal.org/files/projects/mapbox-7.x-2.0-alpha0.tar.gz",
"feeds 7.x-2.0-alpha3" : "http://ftp.drupal.org/files/projects/feeds-7.x-2.0-alpha3.tar.gz",
"geofield 7.x-1.0-alpha1" : "http://featureserver.phase2technology.com/sites/community.aegirproject.org/files/fserver/geofield-7.x-1.0-alpha1.tgz",
"openlayers_slim 1.7" : "http://appserver.openpublicapp.com/sites/community.aegirproject.org/files/fserver/openlayers_slim.tgz",
"job_scheduler 7.x-2.0-alpha2" :"http://ftp.drupal.org/files/projects/job_scheduler-7.x-2.0-alpha2.tar.gz"
}
7.3.2. Complete Example
This is an example of the Project Mapper App manifest
{
"name": "Project Mapper",
"description": "Open Public's Project Mapper is a great way to present your work on a beautiful map. It provides you with a landing page that shows your projects as pins on a map and as a grid of image thumbnails giving your users a quick way to get an overview of your activities. Each project has its own page for publishing detailed information. You can create new projects by simply logging on to your Open Public Site and filling-out a web form.",
"machine_name" : "openpublic_projects",
"version" : "1.0-beta1",
"downloadable" : "openpublic_projects 7.x-1.0-alpha3",
"author" : "Development Seed",
"author_url" : "http://www.developmentseed.org",
"screenshots" : ["http://appserver.openpublicapp.com/sites/community.aegirproject.org/files/apps/project-mapper-screenshot1.jpg"],
"logo" : "http://appserver.openpublicapp.com/sites/community.aegirproject.org/files/apps/project-mapper-logo.png",
"demo content module": "openpublic_projects_demo_content",
"demo content description": "Pre-loaded content to assist in easier demonstration of the app. Once switched off, the demo content is removed from all areas of the public web site, allowing users to populate the site with their own content.",
"dependencies": {
"openlayers_views": "openlayers 7.x-2.0-alpha1",
"openlayers_plus": "openlayers_plus 7.x-1.0-alpha2",
"feeds": "feeds 7.x-2.0-alpha3",
"mapbox" : "mapbox 7.x-2.0-alpha0",
"geofield" : "geofield 7.x-1.0-alpha1",
"job_scheduler" : "job_scheduler 7.x-2.0-alpha2"
},
"libraries" : {
"openlayers_slim" : "openlayers_slim 1.7"
},
"downloadables": {
"openpublic_projects 7.x-1.0-alpha3" : "http://featureserver.phase2technology.com/sites/community.aegirproject.org/files/fserver/openpublic_projects-7.x-1.0-alpha3.tgz",
"openlayers 7.x-2.0-alpha1" : "http://ftp.drupal.org/files/projects/openlayers-7.x-2.0-alpha1.tar.gz",
"openlayers_plus 7.x-1.0-alpha2" : "http://featureserver.phase2technology.com/sites/community.aegirproject.org/files/fserver/openlayers_plus-7.x-1.0-alpha2.tgz",
"mapbox 7.x-2.0-alpha0" : "http://ftp.drupal.org/files/projects/mapbox-7.x-2.0-alpha0.tar.gz",
"feeds 7.x-2.0-alpha3" : "http://ftp.drupal.org/files/projects/feeds-7.x-2.0-alpha3.tar.gz",
"geofield 7.x-1.0-alpha1" : "http://featureserver.phase2technology.com/sites/community.aegirproject.org/files/fserver/geofield-7.x-1.0-alpha1.tgz",
"openlayers_slim 1.7" : "http://appserver.openpublicapp.com/sites/community.aegirproject.org/files/fserver/openlayers_slim.tgz",
"job_scheduler 7.x-2.0-alpha2" :"http://ftp.drupal.org/files/projects/job_scheduler-7.x-2.0-alpha2.tar.gz"
}
}
8. Best Practices for Creating Apps
The purpose of this section is to describe how apps are created and built. These are best practices that are enforced by the App Server Owner.
8.1. How do you create an app?
Creating Apps for OpenPublic follows the same process as creating modules for Drupal. The difference is that apps are Features-based and KIT-compliant, and often bundle together configuration forms, demo content, and other functionality to ensure that they're simple to implement with a single click. Here's how to do it:
8.1.1. Plan for your App
As you're determining the app you'd like to build, a little pre-planning is key. Consider the following:
- What is the specific problem you're trying to solve with this app? Does this need apply to many people?
- Is there an existing app that is solving this problem already?
- If there is an existing app, why would you not use that? What is it missing or unable to accomplish?
Define your app, so that it:
- Solves a specific problem
- Solves it completely
- Does not solve anything else
For help with planning your app, the community.openpublicapp.com forum may be a great place. Use this opportunity to refine your plan and ensure that there is a need in the community for it.
8.1.2. Create or locate the main module for your app
Creating the main module for your app is not unlike general module creation for Drupal. Here are a few specifications:
- Your main app module should be Features-based. Documentation and instructions for packaging modules as features can be found here: http://drupal.org/node/580026
- Your main app module should be KIT-Compliant. The KIT specification for Features can be found here:
You’ll find the Apps API at apps.api.txt. Apps uses the standard hooks architecture of Drupal for its API. For your app, you’ll need to implement the hook_apps_app_info in the module.
8.1.3. Create or collect enhancements for your app
These are optional elements of an app, but will may be important to ensuring usability, which is important for any app. Consider the following:
- Create a config form that will live in the Apps interface for controlling how your App functions. Using the Form API Quickstart Guide will make this easier: http://drupal.org/node/751826
- Create a demo content module to bundle with your module that does the following:
- When enabled, places content on the site that demonstrates the feature set of your app
- When disabled, removes all demo content
8.1.4. Ensure compliance
Make sure that your app meets the stability, reliability, and compliance requirements necessary for Apps.
8.1.5. Documentation for Submission
Create and compile the necessary submission information (see below)
8.2. The App Specific Hook
This can be found in modules/apps/apps.api.php
In order to provide a more integrated experience to the App console your app should implement hook_apps_app_info. This module allows you to describe certain aspects of your Apps functionality to the App Console. This hook is to return an array with the following information:
- Key
- Notes
- demo content description
- This tells what the App demo content will do. It is placed on the configure form along with the mechanism for enabling/disabling the demo content.
- demo content module
- The preferred way for an app to provide demo content is to have a module that when enabled will add demo content, and when disabled will remove demo content. This module should be a submodule of the App but it is not required. If it is not a submodule of the main App module it should be part of the manifest dependent modules.
- demo content enabled
- If demo content is provided as part of an existing module, then this will be the function call that should return TRUE if demo content is currently enabled, FALSE otherwise.
- demo content enable
- If demo content is provided as part of an existing module, then this will be the function call that will enable/install the demo content and should return TRUE if it was successful.
- demo content disable
- If demo content is provided as part of an existing module, then this will be the function call that will disable/uninstall the demo content and should return TRUE if it was successful.
- configure form
- This should be a function that will return a Form API (FAPI) array that will be rendered in the Apps' App Console configure form.
- post install callback
- This will be a function that will be called after the App is enabled initially or if the app has been uninstalled.
8.3. Submitting an App to an App Server
8.3.1. Where to Submit
Submit your app according to the specific requirements of the distribution.
8.3.2. What You Need to Provide
Submitting should be a tarball with a manifest.app file as well as any image referenced in the manifest
The manifest.app should be an ini file with the following items:
name = Test App
description = Test App is used to show how apps work
machine_name = test_app
version = 1.0
downloadable = test_app 1.0
author = Phase2 Techonology
author_url = http://www.phase2technology.com
screenshots[] = screenshot.jpg
screenshots[] = screenshot2.jpg
logo = logo.png
dependencies[views] = views 3.0-beta3
dependencies[ctools] = ctools 1.0-alpha4
downloadables[views 3.0-beta3] = http://ftp.drupal.org/files/projects/views-7.x-3.0-beta3.tar.gz
downloadables[ctools 1.0-alpha4] = http://ftp.drupal.org/files/projects/ctools-7.x-1.0-alpha4.tar.gz
downloadables[test_app 1.0] = http://ftp.drupal.org/files/projects/test_app-7.x-1.0.tar.gz
- Name
- Name of the App.
- Example: "Project Mapper"
- Description:
- Description that tells what it does and who its for, for the apps listing in the OpenPublic apps server.
- Example: "OpenPublic's Project Mapper is a great way to present your work on a beautiful map. It provides you with a landing page that shows your projects as pins on a map and as a grid of image thumbnails giving your users a quick way to get an overview of your activities. Each project has its own page for publishing detailed information. You can create new projects by simply logging on to your OpenPublic Site and filling-out a web form.",
- Machine name
- Name of the main module.
- Version
- Which version of the app this is. e.g. 1.0-beta1
- Location
- Where the main module is found. e.g. http://drupal.org/files/projects/openpublic_projects _7.x-1.0-alpha3.tar.gz
- Author
- Who created this app; e.g. Development Seed
- Author URL
- Link to the author’s site; e.g http://www.developmentseed.org
- Screenshots
- An array of screenshots relative to the path of the manifest.app. A list of screen shots that will show up in app interface. e.g. http://appserver.openpublicapp.com/sites/community.aegirproject.org/files/apps/project-mapp...
- Logo
- A logo or icon for the app itself; this path should be relative to the manifest.app path
- Dependencies
- List all the modules that need to be installed for the App to work; and any modules that those modules depend on. The key is the name of the module, and the value is the name of the downloadable of which it is a part.
- Downloadables
- List all downloadables and the location from which they can be downloaded. Downloadables keys are referenced from the dependencies
- If you want to test the app locally then refrence the manifest from the module.info:
apps[local][manifests][] = app/manifest.app
-
... and place your manifest and images in a subfolder of the module:
app/
manifest.app
screenshot.jpg
screenshot2.jpg
logo.png
8.4. Additional Best Practices for Building Apps
8.4.1. KIT compliance
As we said earlier, we believe Apps to typically function best when implemented as Features. In addition to being a Feature they should be a KIT-compliant Feature to make sure it does not have incompatibilities with other Apps. At it's basis, KIT is a strategy for naming conventions and component exportable strategies that aim for independence and interoperability. Read more about KIT here:
8.4.2. Demo Content
One of the hardest things about installing new functionality in Drupal is that often times you never know quite how to use it. The demo content functionality of Apps aims to lower the barriers to using an App by provide some default content that can be easily enabled and disabled. Demo content can be implement in any way you see fit. Some options are: * Manually create nodes, taxonomy, menus, etc and track their individual ids to that they can be removed later. * Use Feeds to import/export content from included data files (see: Ideation app) * Use Default Content to give Nodes machine names and export them with Features * Use UUID Features to export content into Features for inclusion.
There are countless ways to crack this nut, the real goal here though is that you can turn this content on AND OFF. This allows a user to play with content to see how it works and when they are done, disable the module and have all of that installed content disappear so they can start building their own content.
8.4.3. Contextual App configuration
Oftentimes with Drupal modules, when you are finished installing you need to track down (if you even know to look) where the modules many configuration forms are. Sometimes they are in admin/config sometimes they are in admin/structure, other times in a more specialized location. The point is that, currently in Drupal, once a module is enabled you need to track down some forms to allow for information gathering and configuration. Drupal 7 makes it a step better by providing a configuration link in the modules page but sometimes you want/need more.
Here is where Apps shine, giving you a configuration form presented immediately after enabling your module and in the context of where you read about, downloaded, and enabled your App. At any point in the future you can return to the App Console and find your app and configure it directly form the console. This config form can be use to gather API keys, set permissions or any other thing might need to do. This configuration destination also natively support the enabling/disabling of demo content.
9. Roadmap for future work by the Open App Standard Working Group
The Open App Standard Working Group will continue exploring concepts around Apps in Drupal, starting with gathering community feedback and interest in the draft here, and continuing to cover more aspects of the Apps ecosystem. The following are a few areas the working group is considering:
9.1. Questions around Monetizing Apps
Of interest to many community members is how to effectively monetize an App, or create an "app market," or market. A few app marketplaces and app servers exist now, and inevitably, more will arise. After learning about the ways community members are building and using Apps, we'll collaborate to define the architecture and best practices around a monetized App market. Among the many questions that will arise, we anticipate the following:
- What is the architecture of an App market?
- How do you create an Apps market?
- Who can create an App market?
- How do third party service providers provide their services through an App market?
- Where are App markets housed, hosted, and located?
- How exactly does an App market transaction take place? Where are the points where money/ code/ services/ permission/license can change hands?
- Who runs an App market?
- Where/when can there be multiple App markets?
- How does each player in the Apps Ecosystem interact with the market?
- What are the potential costs, risks, and benefits to each player in the market?
- What does the App consumer see in the transaction?
9.2. Considerations around Apps in an open source community
Apps, and how they are potentially monetized, in an open source community is of course something that creates questions and challenges. As a group, we’ll seek to address the challenges around the “Apps” concept in an open source community, and to offer best practices around community contribution and participation. Among those questions:
- Does Apps create a "new class" of functionality in the community?
- Can anyone create an app market in a distribution? How?
- What components of the Apps ecosystem are open source?
- Who maintains the open source components of the ecosystem?
- Is there a single "Drupal App Market?" Could there be? What are the community implications?
Software-as-a-Service Business Model Specification
This document has moved to http://community.openatria.com/team/saas-business-model
Universities using Aegir
This page is a placeholder list of all of the universities and other large institutions using Aegir. If you work with Aegir in a university, please add yourself here!
- [UNC Charlotte: Student Union, Activities, and Recreation Department (http://suar.uncc.edu)
- The Ohio State University, College of Arts and Sciences, College of Engineering
- The George Washington University
- The MBLWHOI Library (http://www.mblwhoilibrary.org), Marine Biological Laboratory (http://www.mbl.edu), Woods Hole Oceanographic Institution (http://www.whoi.edu) in Woods Hole, MA, USA.
- College of Life Sciences, University of Dundee, Dundee, Scotland (http://www.lifesci.dundee.ac.uk)
- University of Science and Arts of Oklahoma (http://usao.edu)
- University of Oregon (http://uoregon.edu)
- SHANTI, University of virginia (http://shanti.virginia.edu)
- Oregon State University (http://drupal.oregonstate.edu)
- Aix-Marseille Université (http://www.univ-amu.fr)
- Washington State University Vancouver (http://vancouver.wsu.edu)
- Western Washington University (https://wwu.edu)
- University of British Columbia - UBC Web Services
- Ethiopian Civil Service University (http://www.ecsu.edu.et)