In the early days of IT, only the brave or foolhardy tried to compete directly with industry giants like IBM and...
Microsoft. Today, that industry giant is in the cloud -- and it's AWS. And for technology providers with tools that work specifically in the AWS environment, such as third-party monitoring tools, their existence and often their livelihood has been at the tolerance of AWS.
Recently, AWS has made some moves to indicate it might try to dominate the entire AWS monitoring landscape. But industry watchers say that, given the breadth and complexity of the cloud market, it's unlikely that AWS will choose to crush third-party monitoring companies' business -- or that it even could. Some enterprises use third-party or open source tools as a matter of policy, reducing concerns of vendor lock-in with AWS.
Native AWS monitoring tools on offer
AWS is a one-stop tool provider for not just hosting, but managing and monitoring workloads on its platform. For example, AWS X-Ray, which is available in beta, helps analyze and debug production environments and distributed applications, such as microservices architectures.
Another example is Amazon CloudWatch, a monitoring service for AWS cloud resources and applications that customers run on AWS. CloudWatch has features such as CloudWatch Logs, CloudWatch Events and CloudWatch Alarms to quickly notify administrators of changes to a cloud resource.
AWS CloudTrail shares log files between accounts and provides real time monitoring of calls to application programming interfaces. This comprehensive -- and regularly expanding -- list of capabilities isn't enough to keep enterprise IT shops from deploying third-party monitoring tools for their AWS workloads.
"We use a combination of AWS-native and third-party tools, depending on the need and the applications," said Nick Searle, CTO at MOJO Marketplace, a community of professional designers and developers based in Salt Lake City. "Generally speaking, we do not want to be siloed or locked into a single set of tools from a single vendor -- Amazon or any other."
Amazon's tools vary from minimal to well-developed, Searle noted. So, depending on the need, Amazon tools are not always the best option.
Third-party monitoring tools to consider
MOJO uses Server Density, a global hosted service that monitors public and private cloud environments. Citing disruption from a recent Amazon Simple Storage Service outage, Searle said his organization wants redundancy in terms of tool availability and data. He uses both AWS tools and Server Density to provide an interlocking experience; each tool takes a slightly different approach to monitoring, and they can help clarify or confirm each other's conclusions.
Functionality and features also matter to Searle. MOJO uses Server Density because it doesn't think Amazon monitoring tools provide the level of granularity his business needs.
"Some of Amazon's tools have a pretty steep learning curve," Searle said. "We prefer to work with tools that are easier to use out of the box, yet sophisticated enough to grow with us."
MOJO also relies on Ansible configuration management, which provides a more controllable and customizable experience for IT automation than built-in AWS tools.
Over time, AWS likely will offer more "out-of-the-box" monitoring, said Greg Arnette, founder and CTO of Sonian, based in Waltham, Mass. Sonian has been working with AWS for more than a decade to archive and analyze communications data in the cloud. These tools could come as extensions to CloudWatch or as completely new offerings.
AWS will likely focus on monitoring services and apps that run on its cloud, instead of a multicloud monitoring approach, Arnette added. "Many growing companies have more diverse monitoring requirements, and that is where the third-party monitoring opportunity will exist," he said.
And the cloud juggernaut may not be able to sell everyone on its native AWS monitoring tools anyway. Engineers like to build their own monitoring tools to target specific needs, and there already are several open source third-party monitoring tools. In addition, closed and open source products, from New Relic to Sensu, offer enterprises several monitor resources to choose from.
New deployment models affect third-party monitoring tool technology
The tagline for many products in the application performance monitoring (APM) market was once "from URL to SQL," according to Cameron Haight, a research vice president at Gartner. But that's not what Amazon provides today in terms of breadth.
With more cloud-native architectures in play, such as containers and microservices, "complexity is on the outside, not the inside," Haight said. Therefore, developers that design around microservices need to focus on how those microservices connect with one another. And that means capabilities like tracing become more important than application monitoring.
With the growing use of microservices, some vendor pricing models are not oriented toward "a world where containers come and go constantly," said Nancy Gohring, a senior analyst at 451 Research.
Gohring said AWS' app support is somewhat modest for tools like AWS X-Ray, but the limitations are starker with hybrid and multicloud deployments. IT organizations using Google Cloud Platform, Microsoft Azure and other cloud hosting services, or on-premises data centers in addition to AWS likely won't find sufficient breadth from AWS monitoring tools alone.
"AWS isn't seeking to be a grand integrator of monitoring, they only have a slice," she said. "You still need classic APM."
AWS monitoring tools come up short