Andrea Danti - Fotolia

Amazon job listings offer many AWS roadmap clues

Amazon is hiring thousands of people, and where it’s hiring offers clues about where the AWS roadmap is headed.

Hints about Amazon Web Services' product roadmap can be gleaned from the many jobs Amazon has posted for its cloud, including a long-term focus on database services.

Amazon's hiring spree for its Web services division was publicly discussed this month by CTO Werner Vogels, who said 30% to 40% of many engineers' time at the company was being taken up by hiring. The company has more than 2,400 jobs within AWS listed on its website.

Some of the job descriptions offer a glimpse into AWS roadmap plans, such as a new cost management dashboard that's in the works and for which Amazon seeks software developers.

The new cost management console will provide tools such as a dashboard and graphical reports to make it easy to understand bills and visualize historical consumption patterns, according the AWS job posting, titled Software Development Engineer – AWS Cost Insights.

This tool will also enable forecasting, budgeting, and alerts so customers can proactively monitor and manage future spend, scale to support the largest AWS customers, and provide external APIs so customers can write their own cost analysis tools if they prefer.

Currently, AWS offers Trusted Advisor, which can point out things such as under-utilization of resources to help cut costs. It also has a Cost Explorer tool included in the AWS Billing and Management Console, but the tool only displays data from the current month and the previous four months. Cost Explorer also updates expenditure data once every 24 hours.

"Customers at re:Invent last year said that was something they were looking for – better cost analytics and cost management, that they felt like they were lacking," said Carl Brooks, analyst with 451 Research based in New York. "It wouldn't surprise me if they were going to release a product around that somehow."

There is a fairly large partner ecosystem that has been built up around AWS cost management, with companies such as Cloudyn and Cloudability, but "they're clearly filling a hole that Amazon has and Amazon has a well-known pattern of watching those holes and filling them up," Brooks said.

Cost management is also among the tasks an AWS development team known as "Kumo" is charged with, according to the job postings. Trusted Advisor is listed on the AWS website as an example of an application the Kumo team has already developed. The AWS Kumo development team is working on three different greenfield projects, including customer-facing applications. Some of the challenges facing the team are in big data, social apps, machine learning and data mining, according to the AWS job postings. The job description also states that Amazon is filing patents in this area.

While data mining and machine learning could become customer-facing applications, the Kumo team is also focused on improving AWS customer support, based on a Kumo senior manager job description. One data analytics project that's been proposed for AWS billing is integration with the company's Kinesis data streaming service, to provide up-to-the-minute cost information.

AWS roadmap includes S3 index service development

Another set of job postings refers to Simple Storage Service (S3) Index Services. "Customers need highly scalable index services to add and retrieve data from Amazon S3 storage," one posting for a software engineer said. "We need experienced developers who can help build the distributed systems that manage our object lookup data paths."

S3 already has a lookup system by object ID, said Mike Matchett, analyst with the Taneja Group, Inc., based in Hopkinton, Mass.

"S3, like many other object storage systems, can be thought of as a key-value store, like many NoSQL variants out there," Matchett said. It also has some custom headers, which are basically metadata, to be stored with each object as well.

Indexing can then theoretically be attempted at three levels: object key, metadata and object content.

Today, "if you want to index metadata or object content you are on your own with S3," Matchett said.

Users sometimes create their own metadata database, according to Matchett. S3 does some aggregate search indexing of a bucket's keys on the back end, and in the console GUI, users can make use of "auto-complete" when looking for a specific object key, which would depend on the keys being indexed.

This key indexing capability might expand to supporting PERL-like searches on object keys, Matchett said.

"If there is any S3 development it is likely in this area, knowing that some buckets could contain millions of keys and by enabling a more direct retrieval AWS could avoid customers doing hideously inefficient sequential search and filtering operations," he said.

At the same time, S3 is not likely to suddenly evolve data or content level indexing, Matchett said.  

Meanwhile, Amazon is also hiring to improve the overall performance of S3.

AWS roadmap includes database focus

Of more than 2,400 job openings returned on a search for "Amazon Web Services," 960 results are returned when searching on "RDS" for the AWS Relational Database Service. Most of these are in software engineering, and several include this tantalizing tidbit: "We believe that both AWS and Database Services business are in their infancy. Over the next few years, together they will represent a very large business for Amazon."

There are also a number of jobs in Professional Services; don't be surprised if these two categories combine into more Oracle-like support for Amazon's database offerings, 451 Group's Brooks said.

The AWS CloudFront content delivery network is also the focus of 146 job search results. Some industry watchers wonder if AWS is looking to match features offered by rival Akamai Technologies, Inc., which offers things like DDoS insurance and web application firewalls with its CDN.

Others say this is a point of emphasis because CloudFront point of presence locations offer clues as to where AWS might set up its next data centers. Also worth noting along these lines is an opening for an AWS Evangelist in Paris. Other CloudFront hot spots include Seoul, South Korea, and Madrid.

Further tidbits mined from AWS job listings include:

  • A new AWS service in the high-growth, cloud-hosted digital media field.
  • A next generation log database described as a system that has a distributed transactional log as its cornerstone. Amazon has mentioned the integration of CloudTrail and AWS Lambda in analyst calls this quarter, which might be what this refers to.
  • A version one security service developed to help customers protect their data in AWS.

AWS: Too big to hide

AWS generally plays its hand very close to the vest, but that may change as the company prepares to break out AWS results from financial earnings statements in its next quarterly earnings release next month and continues its hiring blitz.

"I'd be curious what these jobs are paying and what that means in terms of the investment picture for Amazon overall, and I think that may be a reason they're going to disclose AWS revenues," said Brooks.  "It's a significant investment they've made in human resources, and if they don't explain that, shareholders might get a little itchy."

If Amazon can justify its hiring spree by showing the growth it's seen in its cloud services, that could stave off shareholder anxiety, though disclosure in general is not seen as in character for the company.

However, "they're reaching a point of size and scale where they realistically can’t do that anymore," Brooks said.

AWS did not comment for this story as of press time.

Beth Pariseau is senior news writer for SearchAWS. Write to her at [email protected] or follow @PariseauTT on Twitter.  

Dig Deeper on AWS database management