Windward in the Clouds

Amazon is on the vanguard of the new cloud computing marketplace and, while I've been EC2-aware for months, until recently I've not actually gotten my hands on it. That changed about a week ago when I decided to see if I could bring up a Hadoop cluster. My longer term goal is to run some scalability tests of the Mahout clustering code and since I lost my little cluster when I left CollabNet I need a replacement.

The economics are really super: ten cents an hour for a box in one of their datacenters and fifteen cents per gigabyte per month for storage. A rather large run of an hour on twenty boxes costs $2, plus the storage costs. I figured I can afford that, so why not see what it takes?

The process was pretty simple. After signing up with Amazon Web Services and downloading their toolkit, I followed their excellent getting started tutorial and pretty soon had a Fedora 8 box running under my control. Getting Hadoop installed required a bit more work as the box comes with nothing but Linux on it. A couple of 'yum' installs later the Java environment was running and Hadoop was installed. I brought up a single node Hadoop cluster and then decided to wait for Hadoop 0.17 to release as it has some DNS optimizations that make running on EC2 simpler.

Since I had a little experience, and since Windward is in dire need of website rebranding improvements I decided to bring up a copy of the Liferay Enterprise Portal to see what that would be like. That required running their installation as well as installing MySQL. There were some script problems with the Liferay 5.0.1 RC script, but in all the process has been moderately easy.

Today I spent most of the day customizing the site with some graphics and an initial page layout. I still need to finish the rebranding and finally point my existing website URL to it, but Windward is now living in the clouds. Woo hoo.