My post on mass hosting actually had a related topic that I ended up putting in draft mode because it got to be too long: how virtualization ties into hosting these days, especially as it relates to web apps.

We announced Bryght Hosting a while back, with just a few details. With our infrastructure partners at Firebright, we're going to make more bundles of services available based around a combination of the Xen virtualization platform and some new management tools that Firebright is developing. Lack of those tools is one of the reasons that IBM mentioned in their hosting and deployment article, where they describe using VMWare. A large part of these tools from Firebright (codename Rack Fire) will be open source and available to anyone interested in managing many virtual instances...but I digress -- go bug Firebright about that, their new site should be up in a couple of weeks.

IBM's write up is quite good, covering some of the basics around virtualization and why you would want to choose that. In the bad old days, a lot of virtual servers were "soft" virtual servers, where resources on a physical system were oversold. Typical Xen environments are "hard" virtual servers -- even if an environment doesn't use all the memory, disk space, etc. allocated to it, those resources are still fully allocated. Much more like really having your "own" server.

One of the first benefits is reliability. Say you're sitting on a piece of real hardware. A dedicated server. And the hardware goes bad. What happens next? Scrambling for backups and hoping for a new piece of hardware available. With virtualization (caveat: there are many ways to do this, and I'm handwaving around bits....), you can take the image of your entire environment, and move it over to another running Xen host and voila! you're still up. Backing up those server images, or even doing a trial upgrade on a complete clone of your production system -- all possible with a virtualized infrastructure.

Another benefit is scaling. Kris Buytaert (no relation to Dries Buytaert, although he is a friend) pokes some holes in the IBM choice of VMWare -- he likes Xen better, but also points to issues with scaling. I suspect that Kris tends to work with REALLY large deployments, and poking around further in his Automating Virtual Machine Deployment paper, it sounds like it is more focused on server consolidation than offer lots of small(er) hosted environments to lots of different people. He has some really good points about how you need to manage large scale deployments around updates between machines. Process becomes really important, so having a tuned, best practices environment where your processes are documented is very important.

So, virtualization and scaling. First off, you can start your site off with a little corner of a big machine. If traffic grows (and, one would hope, revenue) you can increase the size of your virtual server just by upgrading to a bigger piece of that machine. This takes maybe 5 minutes at most -- you "shut down" your virtual environment, it's adjusted to have more memory/diskspace, and then you bring it online again. No moving between servers, and pretty painless.

Of course, this only "scales" as big as the piece of physical hardware you're on. I'm not going to get into grid systems right now, but some of the ideas there are that you can scale incrementally, buying capacity as you need. We're cooking up something similar with Firebright: virtual clusters. For customers that want increased redundancy and/or future scalability, rather than starting on a small, single instance with Web + DB on one box, we can deploy a small cluster of virtual instances --- maybe 2 web front ends and a database backend, or an out-of-the-box redundant setup, with 3 web front ends and 2 database backends. Need to scale? First, the individual components can be allocated more resources -- from small web and db to medium web and db, etc. Next, since the whole setup has been configured to scale horizontally, you can spin up additional web and database nodes as needed. Caught in a Digg-storm? Add a few virtual environments temporarily, and then turn them off again when the traffic levels go back to normal.

I finally got around to stopping by the IBM developerworks forums and leaving some feedback for the folks that are doing the great series of articles on developing a site with Drupal. They're diving all the way down into developing modules, including explaining Drupal's hook system, which is great to see.

[Part of a series about some of the cool people I met at OSCON and Portland 2005 Drupal Conference]

Jonathan from Firebright is razor sharp technically from both a business and technical sense. And it doesn't hurt that lots of our other views align too. It was a pleasure to hang with him (over far too many beers) at OSCON and DrupalCon.