Choose and manage a public cloud storage service successfully
A comprehensive collection of articles, videos and more, hand-picked by our editors
High-performance applications come with unique storage challenges. Certain AWS services and features, such as Amazon...
Elastic File System and Elastic Volumes on EBS, address storage complexities to simplify management and increase flexibility. Each tool has specific benefits over the other.
Elastic Volumes, a feature of Elastic Block Store, improves AWS storage management for high-performance block stores. But Amazon EFS offers more flexibility over file storage. And while EFS might cost more than EBS, some customers will find that it better suits their AWS storage management needs.
EBS gives IT pros better control over the performance of storage attached to Elastic Compute Cloud (EC2) instances. However, EBS comes with management overhead; Elastic Volumes reduces some management headaches of tuning system performance. EBS provides the equivalent of a raw hard drive, and developers can use Elastic Volumes to tune the file system to optimize highly performant apps.
Amazon EFS creates some flexibility and has a good pricing structure to use for write-intensive apps, but it has less configurability than EBS. EFS has a virtual file system that mounts to multiple EC2 instances; this means that thousands of EC2 instances can connect to an EFS instance. Enterprises can also connect EFS to their on-premises data centers. EFS supports some level of redundancy, and developers can access it across multiple availability zones. Dev teams also can configure EFS for parallel access to support faster throughput than EBS. But EFS latency can be higher than EBS, and the service is less customizable.
Amazon Simple Storage Service (S3) provides greater reliability, flexibility and ease of management than either EBS or EFS. But while S3 is an ideal AWS storage management choice for many workloads, the costs can be prohibitive for write-intensive applications.
Does the Amazon S3 outage indicate security vulnerabilities?
Last March's AWS outage wasn't caused by hackers or a cyberattack, but the incident may have security implications for the cloud giant. Check out this podcast for the latest information on potential Amazon S3 vulnerabilities.
Customize EBS volumes on the fly
The EBS system architect is the closest to a bare-metal design on AWS, providing a bare hard drive that resides in physical proximity to the EC2 instance. Developers customize and optimize EBS volumes to improve applications. It can be an ideal choice for computationally intensive applications.
AWS offers four volume types for different use cases: General Purpose SSD (gp2), Provisioned IOPS SSD (io1), Throughput Optimized HDD (st1) and Cold HDD (sc1). Gp2 provides a balance of price and performance for a variety of transactional workloads, while io1 offers the best performance for mission-critical applications. Frequently accessed throughput-intensive workloads benefit most from st1 volumes, and sc1 provides the lowest performance at the best cost.
Until recently, ops personnel had to take a snapshot of the volume to make any sort of changes. And they still need to take a snapshot for disaster recovery purposes or to shrink a volume when it's larger than required. The new Elastic Volumes feature enables ops to expand and change storage performance characteristics of EBS volumes, such as volume types and IOPS capacity, without taking them offline.
EBS pricing ranges from $0.025 to $0.125 per GB-month of storage with an additional charge for more IOPS on the io1 volumes.
EFS provides better AWS storage management
EBS sits on a physical system that requires regular backups to address hardware failures. Prior to this, developers had to shut down, copy and restart volumes to change instance sizes or performance characteristics.
AWS released EFS in 2016 to address some of EBS' management headaches and challenge a host of third-party services that try to address those same problems. EFS works as a virtual file system that's accessible from any availability zone and even from within the enterprise using AWS Direct Connect.
Multiple EC2 instances can directly connect to a single EFS instance. But only one EC2 instance can directly connect to an EBS volume. Other EC2 instances must communicate with a host that's directly attached to an EBS volume to access the data. But the physical proximity of an EBS volume and the host machine lowers latency for I/O-intensive apps.
Developers can configure EFS for parallel access to support throughput speeds of multiple GBs per second, compared to speeds of about a GB per second for EBS. In addition, an EFS instance distributes across availability zones for redundancy. EBS provides better maximum performance when data processing occurs on a single EC2 instance. But when an enterprise needs to process data on separate EC2 instances in parallel, EFS provides better performance.
EFS charges $0.30 per GB-month of storage. Standard S3 storage is $0.023 per GB-month but adds $0.005 per 1,000 write requests and $0.0004 per 1,000 read requests.
EBS with Elastic Volumes is probably the best bet for write-intensive applications running on a single EC2 instance, such as data analytics. Developers can use Elastic Volumes to aggregate large volumes of data at high speeds and then tune down performance characteristics of the volume type to reduce costs when performing analytics. Elastic Volumes could also allow a retailer to turn up the IOPS at peak times and then turn it back down after the rush.
EFS is probably the better option in cases when per-operation latency is suitable for an application. The service provides better automated management than EBS. It also offers redundancy and works with applications that run on multiple EC2 instances. EFS is best suited for big data and analytics applications, media processing workflows, content management, web serving and home directories.
Use these tricks to improve AWS storage performance
See how IOPS affects AWS storage
Amazon EFS isn't for everyone