A Blog

There has been some buzz about Microsoft’s Windows HPC offering. Now, I’m not at all a fan of Windows. I’ve mostly not ever been happy with it’s performance on my desktop and usually laugh at the trials people running Windows servers have to go through. Though, maybe, Microsoft can make a difference with this product.

Where I work, people generally buy into really large clusters. Their nodes are added into the queueing system and they run jobs. Sometimes if people have very special needs, they get their own cluster. The big idea behind my group making big clusters is that researchers can share the resources they’ve bought and the big pie in the sky can be much larger.

I think aggregating resources is a great thing, but scaling every little piece of the infrastructure to handle a thousand node cluster does not seem to be easy. Network gear that can provide great performance to that many nodes is expensive. Cooling that much gear is difficult in old data centers. Powering that much gear is just a headache. Even making sure the NTP server can handle the load needs to make a quick stop on the neuron trail in the minds of admins.

Scaling clusters to large numbers is expensive. That’s why I wonder if Windows HPC can make a small dent. It can bridge that gap between having fast, cheap hardware and no one to run it. Many people “use Ubuntu daily,” but it seems few undergrad/grad students really know how it all goes. Sure, maybe a Professor can find a knowledgeable student to make a Linux cluster go.. But, students leave, and students with Linux knowledge to that depth are rare.

I wonder if for the crowd that needs a “small” amount of cores or prefers dedicated access to their resources, if Microsoft may actually come through to shape the future to come. Or, they may have just unleashed the next generation of botnet…