A Blog

What is cloud computing? To me, it is when hardware or software is run someplace “else” for you. At Purdue, it appears we have interest in doing this for researchers with computing and storage needs. But, I have been doing “computing in the cloud” for a long time before getting involved with Purdue’s effort..

Tinkergeek.com has existed in one form or another for quite some time. Usually, it is hosted in a virtual machine running in someone else’s data center. Over the years, one or both of the machines behind Tinkergeek have been hosted with Linode, Slicehost, or a now defunct Unixshell. I’ve greatly enjoyed having accounts with both Slicehost and Linode. For $20/month or the like, they have provided very solid performance and uptime. To save costs and not host all my data entirely in cloud, the secondary machine for Tinkergeek is hosted at home on my DSL modem.

The next service that can really be defined as cloud computing is my Google Apps hosted domain. It provides my Google Calendar, gMail, and Google Docs access. Both calendar and mail have been very stable since I moved into Google over a year ago and after going with the “Premium” service, I have not had a single hiccup. Of course, even though I pay Google for service and “technically” have a phone number to call if everything goes to hell, I also make a local copy of all my mail and calendar data..

For entertainment, who wants to set their schedule according to some network’s broadcast schedule? A lot of people I know use TiVo devices to get time-shift their television watching. Running a dvr off of over-the-air broadcast is a pain in the rear, so I just use Hulu. This is probably the only “free” cloud service I use, although it does make me watch commercials. No big deal, and it has been quite nice to be able to watch just the shows I care about whenever I want to view them.. Although, Big Media seems scared by Hulu and keeps playing games with the availability of show episodes, making this definitely not the most reliable service on the planet.

A while ago, I attempted to use Amazon’s S3 service for storage in the cloud. I never found a very convenient interface for pushing and retrieving files from S3 and eventually stopped using the service. Then, DropBox came out. It is backed by S3 for storage and provides an excellent interface that integrates very well into OS X. Plus, even though I am using the free-account, DropBox syncs all my data to the machines joined to my account. So, even if Amazon goes down, my data is still available. I like that automatic insurance.

So, there is how I do my computing in the cloud. Nothing very funky or very far out there. Webmail, shared web hosting, and file services (like Xdrive) have been around since before I even got my first computer; now its all just called “cloud computing.” Maybe its the pretty, dynamic Web2.0 interfaces on everything that makes computing deserve to be high in the clouds?

From Amazon to Linode and to scientific clouds, everyone seems to love using Xen to run a service for others. It appears the open source and easily scriptable nature, not to mention the lack of licensing problems and cost, make this a compelling option when you need to host a lot of VMs. On the other hand, it appears VMware has made highways over Xen when it comes to mainline IT and the desire to decrease cost. Nothing beats a manager’s opinion after getting flogged by the VMware sales’ stick.

After running VMware Server 1.0.5 for work’s small physical server to VM effort and deploying Xen for a scientific cloud, I have started to look into what’s next..

VMware Server 1.x is getting old and support has begun to ware thin. So, I hitched up an Ubuntu box with VMware Server 2.0 and started poking around. The first major difference is the command and control interface; it is now a web page with a lot of javascript and a plugin to view machine consoles. This is quite a bit different from the native X11 based interface that “just worked” over SSH with X forwarding. I could not for the life of me get the plugin working on my Mac natively.

The next stop for this train was more free VMware products: VMware ESXi. I had deployed an evaluation copy of ESX before and was taken aback when the only way to manage ESXi was from a Windows-only client. After some digging, to truly manage ESXi installations, you need a Windows-based server, only then could you get even a web interface.

It just seems wrong to rely on a Windows box to manage a fleet of VM-ized servers.. VMware Server 1.x was still looking pretty good.

Then, I remembered the Xen folks were doing a lot of work toward supporting unmodified guests. The Xen HVM stuff looks very promising. It does require having a recent processor and maintaing a primary host operating system, but allows one to have fine grain control over the network bridges and be completely open source. Using Xen-tools it is nearly trivial to deploy a new Linux distribution in an image and giving just a couple of parameters in a config file, you can boot Windows. Getting an encrypted VNC session for a console is a whole lot easier to deal with than attempting to get some sort of silly browser plugin functioning.

Now, just to figure out if sacrificing VMware image compatibility is worth abandoning VMware’s rather ridiculous management concepts for a truly free choice.

What is Cloud Computing? It appears that computing in the cloud as defined by Amazon is Virtual Machine Hosting on Demand, along with a whole host of storage options. According to the Globus Nimbus Project, computing in the cloud is about being able to deploy a computing infrastructure on demand at a remote site. Out on the, now old buzz word, Grid.

After deploying Nimbus on a small cluster of reused machines at Purdue and supporting it for a bit, I think this cloud computing thing is pretty cool for some things. A person can just blink and have a machine up and running. Although, there are a few problems..

Networking can be quite a problem. Every site has their own rules and restrictions, though Purdue has a pretty unrestrictive set of rules. As well, a person deploying a VM at some other random site has no control over what IP the machine gets or what performance the network can provide. Again, at Purdue this is generally not a problem as machines that get spun up land in space that can go out over a number of high-speed research networks. Though, deploying a VM “blind” at a remote site, one never knows what speed the network is, how the site is connected to the Internet, or how oversubscribed the switch is where their VM lives. I assume Amazon offers a fair amount of network performance for “web applications,” but spinning up a high speed cluster in the mystic cloud can be hindered by the hidden unknown.

Now, how about all that storage? Well, there is another pile of hidden unknowns. Obviously, to spin up a VM from an image on a host, the image will have to be transfered. The nice bit about the Nimbus software, it supports images that have been gzipped, which can very nicely make the process quick. However, after running for quite some time, a VM’s disk image can quickly balloon to many gigabytes. Moving that around at start up and shut down can take some time. Then, there’s the performance of the disk underneath a deployed VM. Does the host have a redundant RAID? Does it support any amount of transfer speed? What happens if something goes wrong and the image does not get returned? How does one replicate the data in a cheap manner to prevent loss of work?

Finally, Amazon certainly has some amazing level of spare capacity. Need ten machines? Fifty? They probably have the resources available to bring up a couple hundred VMs in a few minutes. This is partly because they have lots of hardware and capital for more hardware but also because they pack multiple VMs per machine. If one’s goal is to support high performance computing for other groups, you cannot quite intend to oversell machines. So, getting a lot of hardware up to support a high performance cloud is not easy. Also, it is hard to keep the hardware going if there is no demand for it.

Finally, how can one pay for supporting such a service? Well, getting it off the ground is easy when one has old hardware from grants. It is also a good thing when one does not have to pay the powering or cooling bills for running old hardware. Then, there is the fact that the networking infrastructure is also nearly free for some numbers of machines under a hundred.

So, do we ask for money for each hour a VM is running? Per hour? Per processor? Per megabyte? Network usage? Wow, all of this seems very 1970′s; paying for each character entered..

At the moment, grant money is always good.