Select Page

This is a guest post by Gilad David Maayan.


More and more businesses are choosing to store their data in the cloud because of the increased accessibility, high performance, and reliability that it offers in comparison to onsite storage. While a number of providers are available, Amazon is a consistent leader among providers, partially due to its wealth of services. 

If you are already using AWS, it makes sense to take advantage of the easy integration of new services and if you aren’t, the availability of options might be what you’re looking for.

AWS Storage Types

Amazon offers several storage services depending on your needs━Elastic Block Storage (EBS) Elastic File System (EFS), Simple Storage Service (S3), Glacier and Storage Gateway. Of these services, the first three are the most relevant and frequently used as Glacier is meant only for the cold storage of archive data and Storage Gateway acts only as a backup center that is connected to your on-premise storage.

All three services are compatible with Elastic Cloud Compute (EC2) instances, which run applications on the AWS infrastructure and serve as virtual machines (VMs), but only EBS requires their use.

Elastic Block Store (EBS)

AWS EBS provides low-latency block storage for EC2 instances and can be formatted to support various file systems. It allows the use of a Redundant Array of Independent Disks (RAID) configuration for added reliability and makes use of snapshots for backup and data loss prevention purposes. 

EBS use requires you to select whether you want to optimize your storage for throughput or I/O Operations Per Second (IOPS) according to the type of storage you wish to use, Solid State Drives (SSD) or Hard Disk Drives (HDD). This selection must be made when initiating volumes but can be changed manually if you determine that you’re not getting the performance you anticipated. 

Scaling EBS storage is not as simple as the other services as it requires the addition or subtraction of volumes, the modification of volume size, or some combination of the two. EBS is best used for high volume databases, application hosting and storage, and plug and play devices.

Elastic File System (EFS)

AWS EFS is an automatically scalable network file system that uses the NFSv4 protocol to mirror a traditional hierarchy structure, giving it a low learning curve and making it highly compatible with existing file structures. Files stored in EFS can be simultaneously accessed by multiple EC2 instances across multiple regions, facilitating easy sharing of files and simplifying collaboration.

Since it allows multiple instances to connect at once, there is less need for storage redundancy, allowing you to potentially save on costs. It is important to note that EFS is not compatible with Windows EC2 instances, only AWS dedicated instances. EFS is best used for e-commerce sites, high-traffic global content management systems, big data analytics, and media processing workflows.

Simple Storage Service (S3)

AWS S3 provides object storage accessed through APIs and HTTP/HTTPS connections and can be used with AWS CloudFront cache service to serve static web pages without a separate web server or EC2 instance. Since files are served directly through the Internet you can access data from any region. 

S3 includes a built-in query feature to allow you to easily find the data you’re seeking and facilitates file versioning for easy restoration and archiving. It is typically managed through AWS Software Development Kits (SDKs) which are available for the most common programming languages. S3 is best used for media distribution or streaming, static website hosting, version management, big data analytics, and log file storage.

Comparison of Services

See the table below for a comparison of these three services’ base characteristics:

FeaturesAmazon EBSAmazon EFSAmazon S3
Max Storage Size16 TB per volumeUnlimitedUnlimited
Max File SizeLimited by volume size47.9TiB5TB
ScalabilityCan manually scale by modifying attached EC2 volumesAutomatic scalingHighly Scalable but not automatic
Data AccessSingle EC2 instance in single availability zoneUp to 1k EC2 instances or servers concurrently with no AZ or region restrictions1 Million+ connections available over the Internet, includes RESTful API interface
Access ControlManaged with IAM policies, roles, and security groupsManaged with user and group level permissions
Only resources with access to endpoints can access files
Private access managed through bucket and IAM policies
Public access managed with Block Public Access
PerformanceVaries depending on volume storage type; baseline of 3 IOPS per GB of storageUp to 10GB/s  with 3GB/s baseline performance
Allows up to 7k IOPS
Supports multipart upload for objects over 100MB
Allows 3500 PUT, LIST, or DELETE requests per second and 5500 GET requests per second
Availability99.99%Highly available (No Service Level Agreement)99.99%
ReliabilityData stored redundantly in single Availability Zone (AZ)
Uses snapshots as backups
Data stored redundantly across multiple AZs
Uses EFS to EFS backup solution
Data stored redundantly across multiple AZs
Uses versioning and cross-region replication

Use Cases
Application hosting, transactional and NoSQL databases, data warehousingWeb serving, content management, media storage, home directories, database backups, developer toolsWeb serving, content management, media storage, backup storage
Cost*Free usage tier:30GB storage with 2 million I/Os and 1GB snapshot storage
Provisioned SSD:$0.125 per GB/mth storage and $0.065 per provisioned IOPS/mth
General Purpose:$0.045 per GB/mth storage 
Standard storage:pay for data stored, $0.3 per GB/mth and $6 per MB/s-Month for provisioned throughput
Infrequent storage:pay for data stored and each time a file is accessed, $0.045 per GB/mth and $0.01 per GB accessed
Free usage tier: 5GB in standard storage class with 20k GET requests, 2k PUT, COPY, POST, or LIST requests and 15GB/mth data transfer for one year
Standard storage:between $0.021-0.023 per GB/mth depend on total data stored

*Summarized pricing only, for full details you can use the AWS Pricing Calculator

In general, S3 is going to be your cheapest, most flexible, and most easily accessible option, provided you’re okay with slower operations. It’s good for storing backups and facilitating remote collaboration but it comes with complex pricing schedules that can quickly get out of hand.

EFS comes with the simplest pricing scheme and is good at handling massive workloads. The downside of EFS is that it isn’t suitable for small, general-purpose use and it is potentially less secure and less reliable because of the way you are forced to share resources with other EFS clients.

EBS is somewhere in the middle. It doesn’t have the accessibility of S3 or EFS but it’s faster than both. It is highly secure and is the go-to choice for application hosting and databasing but isn’t a great choice for file sharing or collaboration outside of development.

Your best option is likely to use some combination of these three services, customized to fit your needs. Although this is a more complex solution, it will offer you the advantages of each while minimizing the downsides that come with trying to make a system all-purpose.

Conclusion

Choosing the service or services that best suit your needs can be difficult and should be done carefully. Even though Amazon offers flexibility in the selection of services and transfer of data, it is still a time-consuming process that you want to make as simple for yourself as possible. Using this guide, you can begin narrowing down your options and making a workable plan for how to make AWS storage services work for you.


Gilad David Maayan is a technology writer who has worked with over 150 technology companies including SAP, Imperva, Samsung NEXT, NetApp and Ixia, producing technical and thought leadership content that elucidates technical solutions for developers and IT leadership. Today he heads Agile SEO, the leading marketing agency in the technology industry. LinkedIn: https://www.linkedin.com/in/giladdavidmaayan/


Subscribe Subscribe to our newsletter and receive our Python Basics Cheatsheet!