Amazon’s Elasticsearch Service shows promise for users who have long loathed the process of setting up Elasticsearch, Logstash and Kibana clusters in the cloud, but there’s a security snag for some users.
Users have alternatives when it comes to restricting access to the Elasticsearch cluster, either through Identity and Access Management (IAM) roles or IP-based whitelists. However, VPC support would be ideal, according to AWS shops.
“Yes, you can control access via IPs and AWS Accounts, etc. but this still means that all of my private subnet instances will need to traverse their public NAT gateway to communicate with the ES end point,” said one Reddit user of the lack of VPC support. “That [expletive] sucks, and defeats whatever performance/bandwidth benefits I can have with my own internal ES nodes.”
Cloud consultants say some users are taking this in stride, but a significant number of customers will hold back from putting this version of the Amazon ES into production.
“It’s 50/50 whether it meets customers’ full set of needs,” said Patrick McClory, director of automation and DevOps for Datapipe, a provider of managed hosting services for AWS based in Jersey City, N.J.
It’s early in the game for Amazon ES, however, and many customers who are still just getting their feet wet with the service understand that evaluation environments are rarely perfect, McClory said.
“It’s Amazon,” McClory added. “It’ll get VPC support soon, I have no doubt – and it’ll be an easy move to production then.”
Meanwhile, without the ability to house Elasticsearch clusters inside VPCs, users have to pay for data transfer into and out of EC2 instances that access the Elasticsearch Service at a rate of $0.01 per gigabyte. It’s a paltry charge at first glance and Amazon doesn’t charge for transfer in and out of ES itself, but those costs can add up, according to Theodore Kim, senior director of SaaS operations for Jobvite Inc., a talent acquisition software maker in San Mateo, Calif.
“On my ultimate wish list would be the ability to run Elasticsearch within our VPC,” Kim said. “I’m hoping that will eventually happen as it did with S3.”
Also, while Elasticsearch clusters and the Kibana plugin can be accessed with a few clicks, the “L” in the Elasticsearch / Logstash / Kibana (ELK) stack will still take some doing for Amazon ES users.
A Logstash plugin must be built, downloaded and installed into DynamoDB by the user, according to Amazon’s documentation. The service also supports only one Logstash output plugin, according to Amazon’s Developer Guide.
Finally, Elasticsearch clusters today are limited to a maximum of 10 nodes per cluster, according to Amazon’s documentation.
Amazon declined to comment on record for this post.