News Stay informed about the latest enterprise technology news and product updates.

Amazon Elasticsearch Service still sanding down rough edges

Amazon’s Elasticsearch Service shows promise for users who have long loathed the process of setting up Elasticsearch, Logstash and Kibana  clusters in the cloud, but there’s a security snag for some users.

The Elastisearch Service (Amazon ES) as it exists today is without Virtual Private Cloud (VPC) support, which has been a source of angst on more than one AWS Reddit thread this month.

Users have alternatives when it comes to restricting access to the Elasticsearch cluster, either through Identity and Access Management (IAM) roles or IP-based whitelists. However, VPC support would be ideal, according to AWS shops.

“Yes, you can control access via IPs and AWS Accounts, etc. but this still means that all of my private subnet instances will need to traverse their public NAT gateway to communicate with the ES end point,” said one Reddit user of the lack of VPC support. “That [expletive] sucks, and defeats whatever performance/bandwidth benefits I can have with my own internal ES nodes.”

Cloud consultants say some users are taking this in stride, but a significant number of customers will hold back from putting this version of the Amazon ES into production.

“It’s 50/50 whether it meets customers’ full set of needs,” said Patrick McClory, director of automation and DevOps for Datapipe, a provider of managed hosting services for AWS based in Jersey City, N.J.

It’s early in the game for Amazon ES, however, and many customers who are still just getting their feet wet with the service understand that evaluation environments are rarely perfect, McClory said.

“It’s Amazon,” McClory added. “It’ll get VPC support soon, I have no doubt – and it’ll be an easy move to production then.”

Meanwhile, without the ability to house Elasticsearch clusters inside VPCs, users have to pay for data transfer into and out of EC2 instances that access the Elasticsearch Service at a rate of $0.01 per gigabyte. It’s a paltry charge at first glance and Amazon doesn’t charge for transfer in and out of ES itself, but those costs can add up, according to Theodore Kim, senior director of SaaS operations for Jobvite Inc., a talent acquisition software maker in San Mateo, Calif.

“On my ultimate wish list would be the ability to run Elasticsearch within our VPC,” Kim said. “I’m hoping that will eventually happen as it did with S3.”

Also, while Elasticsearch clusters and the Kibana plugin can be accessed with a few clicks, the “L” in the Elasticsearch / Logstash / Kibana (ELK) stack will still take some doing for Amazon ES users.

A Logstash plugin must be built, downloaded and installed into DynamoDB by the user, according to Amazon’s documentation. The service also supports only one Logstash output plugin, according to Amazon’s Developer Guide.

Finally, Elasticsearch clusters today are limited to a maximum of 10 nodes per cluster, according to Amazon’s documentation.

Amazon declined to comment on record for this post.

Join the conversation

1 comment

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.

For those that are upset with these shortcomings, I suggest Qbox Hosted Elasticsearch.  We do offer VPC support.  Even moreso, every person on our support queue is a trained Elasticsearch expert.  With AWS, your support questions will go into their general support queue.

(Full disclosure: I am a founder of Qbox)
Cancel

-ADS BY GOOGLE

SearchCloudApplications

TheServerSide.com

SearchSoftwareQuality

SearchCloudComputing

Close