auremar - Fotolia
We asked IT administrators what they wish they'd known before they started running HPC in the cloud on AWS and other platforms. Here's their best advice on testing, networking performance and more:
- Partner up. Make sure your HPC cloud provider offers more than just a help desk number, said Paul Maher, CTO at Milliman. A regular contact -- someone who acts like a partner -- is invaluable. This is especially important, he said, if you're using the cloud provider's platform to offer your customers a product. "When things go round -- and that's going to happen -- the last thing you want is for your vendor choice to be a problem."
- Test network performance. When you're running HPC on-premises, you know everything is working together because everything is clustered together, explained Mike Beller, CTO at Tradeworx, who runs HPC on AWS. But when you move HPC to the cloud, all that togetherness is the most difficult to recreate. "Network performance is the place where you will most likely see a problem," he said. "So you have to think about it ahead of time."
- Be flexible. Timing is everything, said Dr. Nathan Price, associate director of the Institute for Systems Biology (ISB), particularly when it comes to the cloud. If you're cost conscious, think about when you run things. For example, get the lowest possible "Spot Instance" prices. AWS offers a "bid your own price" option for some features. Price's lab tries to avoid sending data to the AWS cloud on Fridays (typically, a day when payrolls are run) or during the Christmas season.
- Optimize projects. If pricing is an issue -- and when isn't it? -- study how your cloud platform works in terms of pricing and computation power and adjust your projects accordingly, said ISB's John Earls, a graduate student who's done much of the lab's hands-on work with its HPC in the cloud projects. "There are lots of options for database pricing, storage and more, depending on the computation you need to run. So, it makes sense to think it all through ahead of time. You need to be savvy when you're making your choices."
- Build a team. Once you deploy software in the cloud, things will go wrong, so you need to ensure you have an organization in place to deal with problems, warned Milliman's Maher. "Most organizations probably don't have a physical team built that is capable of monitoring and maintaining the software and being the point of contact when something goes wrong," he said. His company had to create a separate group to focus just on the cloud deployment, and in the several years since the launch, the team has grown two to three times in size, underscoring how important that role is.
- Do your homework. Check the availability of machine instances to make sure you will have the number you want when you want them, advised consultant Dan Sullivan. Also, he said, if your project is data-intensive, make sure the I/O performance is sufficient.
- Think outside the box. From architecture to coding to strategy to communication, building software in the cloud requires a different approach than building software on-premises, Maher said. "It takes a lot of learning," he said.
About the author:
Valerie Rice Silverthorne is a writer and editor with nearly 30 years of experience covering business, trade, technology, real estate and lifestyle trends. She was an award-winning business writer for The San Jose Mercury News and was a Forbes Magazine top "30 under 30" journalist. She was the editor of ZDNet.com and PC Week/Inside, and a senior executive editor of PC Week and Electronic Business. She works as a freelance writer from her home in Amesbury, Mass. Contact her at firstname.lastname@example.org.