auremar - Fotolia
As a startup, Thinknear, a Los Angeles-based advertising services company, modified Amazon Elastic Beanstalk to resolve log rotation and scaling issues. By doing so, the company was able to meet the needs of its growing API service, according to John Hinnegan, head of software engineering.
Top priorities for Thinknear were speed to market and being able to iterate quickly. Thinknear chose Beanstalk because it provided a pre-configured starting point for its scaling needs, it had a low learning curve and it tied together Amazon EC2 and auto scaling. "If you have a very small team and you're concerned about speed to market, being able to get metrics, deployment and a bunch of infrastructure for free, Beanstalk is a great option," Hinnegan said.
As the company grew, Hinnegan found it necessary to tune and modify Beanstalk's configuration. To do this, Hinnegan used .ebextensions, a mechanism built into Beanstalk that easily lets you extend or configure an environment to suit your needs.
Thinknear generates terabytes of data and frequently rotates the data into the (Amazon) cloud; elastic Beanstalk's configuration wasn't able to handle the large volume of data. "Using .ebextensions, we installed our own custom log rotate configurations," Hinnegan said. Hinnegan also disabled Beanstalk's default log rotate configurations and installed a process to upload logs from Amazon Simple Storage Service to Thinknear databases. "Beanstalk does have a mechanism to rotate logs but it’s designed with much lower volumes in mind than what we serve," Hinnegan said.
Not every feature offered by Elastic Beanstalk had to be modified for Thinknear's application. Hinnegan found the deployment mechanism offered by Beanstalk to be reliable. "Once you wrap your head around the application versions and the way that works, it actually has a really nice Web UI to drive deployments," Hinnegan said. Also, he said, it has worked well to use the aggregated metrics from Amazon CloudWatch and the accompanying dashboard that polls metrics from across the host.
To eliminate infrastructure issues, and avoid spending unnecessary time trying to diagnose problems, Hinnegan's "trick" is to always measure from the outside in. "If we’re diagnosing any kind of issue with latency, response times or errors, we always start at the load balancer and then work our way back," Hinnegan said. Identifying problems using this method saves the time of diving into the application.
Hinnegan currently runs Apache Tomcat on a Java virtual machine (JVM), configuring the JVM to meet his needs as Java evolves. Hinnegan said that incorporating new technologies, like the new asynchronous connection pool, allows him to make necessary improvements to Thinknear's service.
Now admittedly more familiar with Amazon services than when he started, Hinnegan said a positive factor is the stress-testing conducted by power users on the AWS platform. "Even if you find potential flaws or bugs, there are other people finding ways to work around the problem."
The more Thinknear has evolved, Hinnegan said, the more it has become invested in the Amazon ecosystem. Thinknear is a heavy user of DynamoDB, RDS for databases and ElastiCache for caches. The next project for Thinknear will be setting up a data warehouse in RedShift for its operations team. Hinnegan noted, "We're quite happy on the Amazon platform and we're continuing to invest in tools around that ecosystem."