This is a guest post by Gilad David Maayan.
More and more businesses are choosing to store their data in the cloud because of the increased accessibility, high performance, and reliability that it offers in comparison to onsite storage. While a number of providers are available, Amazon is a consistent leader among providers, partially due to its wealth of services.
If you are already using AWS, it makes sense to take advantage of the easy integration of new services and if you aren’t, the availability of options might be what you’re looking for.
AWS Storage Types
Amazon offers several storage services depending on your needs━Elastic Block Storage (EBS) Elastic File System (EFS), Simple Storage Service (S3), Glacier and Storage Gateway. Of these services, the first three are the most relevant and frequently used as Glacier is meant only for the cold storage of archive data and Storage Gateway acts only as a backup center that is connected to your on-premise storage.
All three services are compatible with Elastic Cloud Compute (EC2) instances, which run applications on the AWS infrastructure and serve as virtual machines (VMs), but only EBS requires their use.
Elastic Block Store (EBS)
AWS EBS provides low-latency block storage for EC2 instances and can be formatted to support various file systems. It allows the use of a Redundant Array of Independent Disks (RAID) configuration for added reliability and makes use of snapshots for backup and data loss prevention purposes.
EBS use requires you to select whether you want to optimize your storage for throughput or I/O Operations Per Second (IOPS) according to the type of storage you wish to use, Solid State Drives (SSD) or Hard Disk Drives (HDD). This selection must be made when initiating volumes but can be changed manually if you determine that you’re not getting the performance you anticipated.
Scaling EBS storage is not as simple as the other services as it requires the addition or subtraction of volumes, the modification of volume size, or some combination of the two. EBS is best used for high volume databases, application hosting and storage, and plug and play devices.
Elastic File System (EFS)
AWS EFS is an automatically scalable network file system that uses the NFSv4 protocol to mirror a traditional hierarchy structure, giving it a low learning curve and making it highly compatible with existing file structures. Files stored in EFS can be simultaneously accessed by multiple EC2 instances across multiple regions, facilitating easy sharing of files and simplifying collaboration.
Since it allows multiple instances to connect at once, there is less need for storage redundancy, allowing you to potentially save on costs. It is important to note that EFS is not compatible with Windows EC2 instances, only AWS dedicated instances. EFS is best used for e-commerce sites, high-traffic global content management systems, big data analytics, and media processing workflows.
Simple Storage Service (S3)
AWS S3 provides object storage accessed through APIs and HTTP/HTTPS connections and can be used with AWS CloudFront cache service to serve static web pages without a separate web server or EC2 instance. Since files are served directly through the Internet you can access data from any region.
S3 includes a built-in query feature to allow you to easily find the data you’re seeking and facilitates file versioning for easy restoration and archiving. It is typically managed through AWS Software Development Kits (SDKs) which are available for the most common programming languages. S3 is best used for media distribution or streaming, static website hosting, version management, big data analytics, and log file storage.
Comparison of Services
See the table below for a comparison of these three services’ base characteristics:
| Features | Amazon EBS | Amazon EFS | Amazon S3 |
| Max Storage Size | 16 TB per volume | Unlimited | Unlimited |
| Max File Size | Limited by volume size | 47.9TiB | 5TB |
| Scalability | Can manually scale by modifying attached EC2 volumes | Automatic scaling | Highly Scalable but not automatic |
| Data Access | Single EC2 instance in single availability zone | Up to 1k EC2 instances or servers concurrently with no AZ or region restrictions | 1 Million+ connections available over the Internet, includes RESTful API interface |
| Access Control | Managed with IAM policies, roles, and security groups | Managed with user and group level permissions Only resources with access to endpoints can access files | Private access managed through bucket and IAM policies Public access managed with Block Public Access |
| Performance | Varies depending on volume storage type; baseline of 3 IOPS per GB of storage | Up to 10GB/s with 3GB/s baseline performance Allows up to 7k IOPS | Supports multipart upload for objects over 100MB Allows 3500 PUT, LIST, or DELETE requests per second and 5500 GET requests per second |
| Availability | 99.99% | Highly available (No Service Level Agreement) | 99.99% |
| Reliability | Data stored redundantly in single Availability Zone (AZ) Uses snapshots as backups | Data stored redundantly across multiple AZs Uses EFS to EFS backup solution | Data stored redundantly across multiple AZs Uses versioning and cross-region replication |
Use Cases | Application hosting, transactional and NoSQL databases, data warehousing | Web serving, content management, media storage, home directories, database backups, developer tools | Web serving, content management, media storage, backup storage |
| Cost* | Free usage tier:30GB storage with 2 million I/Os and 1GB snapshot storage Provisioned SSD:$0.125 per GB/mth storage and $0.065 per provisioned IOPS/mth General Purpose:$0.045 per GB/mth storage | Standard storage:pay for data stored, $0.3 per GB/mth and $6 per MB/s-Month for provisioned throughput Infrequent storage:pay for data stored and each time a file is accessed, $0.045 per GB/mth and $0.01 per GB accessed | Free usage tier: 5GB in standard storage class with 20k GET requests, 2k PUT, COPY, POST, or LIST requests and 15GB/mth data transfer for one year Standard storage:between $0.021-0.023 per GB/mth depend on total data stored |
*Summarized pricing only, for full details you can use the AWS Pricing Calculator.
In general, S3 is going to be your cheapest, most flexible, and most easily accessible option, provided you’re okay with slower operations. It’s good for storing backups and facilitating remote collaboration but it comes with complex pricing schedules that can quickly get out of hand.
EFS comes with the simplest pricing scheme and is good at handling massive workloads. The downside of EFS is that it isn’t suitable for small, general-purpose use and it is potentially less secure and less reliable because of the way you are forced to share resources with other EFS clients.
EBS is somewhere in the middle. It doesn’t have the accessibility of S3 or EFS but it’s faster than both. It is highly secure and is the go-to choice for application hosting and databasing but isn’t a great choice for file sharing or collaboration outside of development.
Your best option is likely to use some combination of these three services, customized to fit your needs. Although this is a more complex solution, it will offer you the advantages of each while minimizing the downsides that come with trying to make a system all-purpose.
Conclusion
Choosing the service or services that best suit your needs can be difficult and should be done carefully. Even though Amazon offers flexibility in the selection of services and transfer of data, it is still a time-consuming process that you want to make as simple for yourself as possible. Using this guide, you can begin narrowing down your options and making a workable plan for how to make AWS storage services work for you.
Gilad David Maayan is a technology writer who has worked with over 150 technology companies including SAP, Imperva, Samsung NEXT, NetApp and Ixia, producing technical and thought leadership content that elucidates technical solutions for developers and IT leadership. Today he heads Agile SEO, the leading marketing agency in the technology industry. LinkedIn: https://www.linkedin.com/in/giladdavidmaayan/
Trackbacks/Pingbacks