Aleksandr Bedrin - Fotolia

It takes more than brute force to beat shadow IT

Think you know how much cloud use is in your organization? Think again. Shadow IT is more common -- and harder to control -- than suspected.

When IT professionals consider AWS, one of the first questions they ask is about shadow IT -- how much is the service being used, and by whom?

The answer to the first question is always profoundly shocking, said David Linthicum, senior vice president at Cloud Technology Partners in Boston. "It's always a rude awakening when guys like me show up and reveal how many people are using the cloud," Linthicum said.

IT professionals tend to grossly underestimate their organization's cloud usage, said David Cope, executive vice president of corporate development and CMO at CliQr Technologies, a cloud orchestration vendor.

"If you talk to corporate IT, most will tell you that they know who's doing what, but they're surprised by the reality," he said.

Answering the second question -- who is using what specific cloud resource -- is tough.

"You have to do discovery across organizations and understand what apps and clouds are out there," Cope said. And unfortunately for IT, there are no shortcuts.

"There are no new laws of physics, no new buttons to push," he said.

Digging through expense reports

At the AWS Re:Invent conference last fall, some consultants suggested looking at expense reports as a way to ferret out developers or line-of-business users submitting around cloud resources. Taking things one step further and instating a policy that says no one gets paid for AWS expenses, is "very effective," said Chris Wegman, managing director for Accenture's AWS practice

But looking for shadow IT in expense reports might not be very effective, said Sebastian Stadil, CEO and co-founder of Scalr Inc., a cloud management platform provider. That's especially true if developers use personal accounts. "The discovery period might be quite long," Stadil said.

Furthermore, looking at expense reports might not reveal everything you're looking to find out, said CTP's Linthicum. "A lot of cloud services are free, or people might be disguising them as something else," he added -- a subscription, for example.

That leaves you with the brute-force method of looking at firewall logs, which list the destination IP addresses and domain names for outbound data flows. "And if you want to be a real jerk about it, you can trace it back to specific IP addresses," to discover the requestor, said Linthicum.

Examining firewall logs is a matter of turning on the logging service if it isn't already, and exporting it to an external tool for analysis, Linthicum said.

Alternately, if your goal isn't to merely discover shadow IT, but also to discover potential security risks, there are services from providers such as Bitglass, which compare firewall logs against a database of more than 4,000 cloud applications and services, as well as their relative risk ratings, said Bitglass CEO Nat Kausik. Using such a service may also speed up the discovery process.

"There's nothing to prevent people from doing it themselves," Kausik said, "except that it's manually exhausting when you're looking at 15,000 or 20,000 sites."

Alex Barrett is editor in chief of Modern Infrastructure. Write to her at [email protected].

Dig Deeper on AWS pricing, cost and ROI