Kinesis could push AWS billing to near-real time

Amazon reps piqued attendees' interest at an AWS Roadshow this week when they detailed how Kinesis could be used to enhance the AWS billing process.

CAMBRIDGE, Mass. -- Amazon Web Services could use its Kinesis data streaming service to enhance its billing process,...

an idea customers hope will come to fruition.

Amazon Web Services (AWS) customers say a billing system based on Kinesis would be a good move for the company, following a data analytics roadshow by one company official who suggested a possible internal use for the data streaming service.

Amazon's streaming data analytics service, Kinesis, could refine AWS' billing and metering system, which involves billions of records generated per hour, said Eric Heikkila, business development manager for data analytics for AWS. The AWS billing system currently uses Elastic MapReduce (EMR), better suited to batch jobs, so bills are current up to 48 hours ahead, since it takes about 24 hours to aggregate data and run the EMR batch processing. Using Kinesis and running custom code on "a massive fleet" of Elastic Compute Cloud (EC2) instances could cut that time down to as little as ten seconds, Heikkila said.

Customers in attendance at the meeting whose businesses are based entirely on AWS were enthusiastic about this idea.

"Being able to manage your AWS environment based on more accurate cost data, knowing where you are in terms of your current bill and being able to marry it up with the price on EC2 at a given point in time, being able to spin up EC2 instances [in response], [and] doing all that while managing your AWS budget is pretty powerful," said Arturo Fagundo, founder of, a startup based in Marlborough, Mass., that performs secure backups between AWS accounts for its customers.

There would still be some work to do to take advantage of finer-grained billing data, Fagundo said.

Another session attendee echoed Fagundo's enthusiasm.

"We're constantly developing and integrating new data sources into our products, so seeing near-real-time costs of our efforts would be helpful rather than a 48-hour delay," said Tanya Cashorali, director of analytics at a Boston-based firm specializing in telecommunications data analysis. "When working in such an agile environment it can be painstaking to go back and check all AWS bills and try to figure out which experiment or process incurred which costs."

EC2 instance tagging advances wanted

Depending on the company, speedier AWS billing is not necessarily critical, according to James Staten, analyst with Forrester Research, based in Cambridge.

"A lot of enterprises are miles away from thinking this way, so this is not something that's necessarily an enterprise value proposition," he said. "But if you're a startup, or a digital gaming company, with a 100% cloud-based service or value proposition, there's significant appeal to this."

Google already provides billing information in 10-minute increments, but Amazon can offer better granularity in its billing with its tagging system, according to Staten.

Amazon still needs to work out some kinks with tagging and could benefit from including automatic tagging or policy-based tagging, added Staten.

Reached for comment after the meeting, an AWS spokesperson declined to say whether Kinesis-based billing is on the company's roadmap.

Beth Pariseau is senior news writer for SearchAWS. Write to her at or follow @PariseauTT on Twitter.

Dig Deeper on AWS streaming and file transfer services



Find more PRO+ content and other member only offers, here.

Related Discussions

Beth Pariseau asks:

What else should AWS do to improve its billing system?

1  Response So Far

Join the Discussion

1 comment


Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: