Amazon Web Services has updated its Elastic Compute Cloud instances with the D2 line, offering more memory, faster...
processors and highly dense storage.
The D2 Amazon Web Services (AWS) are an upgrade to the HS1 line. At the high end, the d2.8xlarge instance comes with 36 vCPUs, 244 GB of memory, 48 terabytes (TB) of hard disk drives, and 10 gigabit per second enhanced networking to take advantage of single-root I/O virtualization to boost performance. D2 instances are available immediately in AWS' U.S. East (Northern Virginia), U.S. West (Oregon), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Tokyo), Asia Pacific (Singapore), and Asia Pacific (Sydney) regions.
"When it becomes available to us we will evaluate for a large search cluster that has growing storage versus compute needs for specific types of indexes," Langley said. "They look interesting enough to research and potentially test."
While not offered for data warehousing yet, these nodes could be well-suited to running under the Redshift data warehouse platform under certain workload conditions, said Calvin French-Owen, co-founder of Segment, a startup based in San Francisco which routes customer data for businesses using Redshift and other AWS platforms.
"Some customers are able to use the basic Redshift nodes, but some others have to size what their clusters are using," French-Owen said. "That would be a key place for it."
Segment stores data in AWS' Simple Storage Service, because its applications can tolerate queuing using NSQ, an open-source distributed messaging platform.
"Each node is just writing to its own disk locally, and if we have any problems we can just add more nodes," French-Owen said. "But if we got into a business where queuing wouldn't be acceptable, D2 nodes sound like they would be a good fit."
Redshift nodes come in dense storage and dense compute flavors; the dw1.8xlarge can yield a maximum cluster size of two petabytes, but each node contains only 16 TB of storage as opposed to the D2 line's maximum of 48 TB per node.
In contrast to the d2.8xlarge, the high end of the HS1 line, the hs1.8xlarge, offered 17 vCPUs, 117 GB of memory, 10 Gigabit networking without the enhanced networking feature, and 48 TB of storage. While HS1 instances were labeled "storage optimized," they also didn't come with Elastic Block Store optimization – the D2 line of AWS instances does offer this. The HS1.8xlarge costs $4.60 per hour, while the d2.8xlarge costs $5.52 per hour.
The D2 line of AWS instances also offers more options than the HS1 line; there are two sizes of HS1 nodes, the hs1.4xlarge, which offers 20.5 terabytes of solid state storage for $3.10 per hour, and the aforementioned hs1.8xlarge. D2 instances are also available in four, eight and 16 virtual CPU sizes, starting at $0.69 per hour for the d2.xlarge. Amazon claims the d2.xlarge can achieve 437 megabytes per second throughput to storage, while the d2.8xlarge is said to perform at 3.5 GB per second for reads when used with a Linux Amazon Machine Image.