Core Cloud Architecture · Part 5 of 13
Storage Fundamentals: Object, Block, and File
Three kinds of storage, each suited to a different access pattern.
Instances are meant to be disposable: one can disappear and be replaced without losing anything important, as long as the data it was working with lives somewhere durable. That somewhere is storage, and cloud storage comes in three shapes, each matching a different way of reading and writing data.
Object storage
Object storage stores data as objects. An object is the data itself plus metadata that describes it (content type, last-modified date, custom tags), identified by a unique key (a name) within a bucket (S3, Cloud Storage) or container (Azure Blob Storage). The service treats the data as opaque bytes, so an object can be an image, a backup, a log file, or a model artifact. Objects are read and written through an HTTP API instead of a mounted filesystem. The namespace is flat: the "folders" in a console are a display convention over keys that contain slashes. On S3 and Cloud Storage an object is replaced as a whole instead of edited in place, which suits anything read and written as a complete unit, at large scale and low cost: images, documents, exported datasets, trained model artifacts, and backups. The main offerings are Amazon S3, Google Cloud Storage, and Azure Blob Storage.
Block storage
Block storage presents raw, fixed-size blocks of storage to a single virtual machine. It behaves like an attached hard drive. It's what a VM's operating system and a database's data files sit on, formatted with a normal filesystem, supporting random access and in-place writes the way a local disk does. Block storage is fast, but it's normally attached to one instance at a time, and it's the storage layer underneath many managed databases even when the service hides that detail.
File storage
File storage provides a shared filesystem that multiple instances can mount and access concurrently over a standard network filesystem protocol (NFS is the common one), with the same nested-folder structure a local filesystem has. It's the right fit when several instances need to read and write the same files at once. The providers' documentation lists shared persistent volumes for containers (all three), high-performance computing and batch compute (Google Cloud Filestore, Azure Files over NFS), web serving, content management, and home directories (Amazon EFS). All three also serve applications written for a standard filesystem interface, and Azure's documentation names lifting and shifting such applications as a use case.
| Object | Block | File | |
|---|---|---|---|
| Unit | Objects (data plus metadata), addressed by key | Fixed-size blocks | Files in folders |
| Access | HTTP API | Attached to one VM, as a disk | Network filesystem, mountable by many |
| Concurrent access | Many readers/writers, whole-object replace | Typically one VM at a time | Many readers/writers, file-level |
| Typical use | Media, backups, datasets, model artifacts | VM disks, database data files | Shared uploads, legacy POSIX workloads |
| AWS | S3 | EBS | EFS |
| GCP | Cloud Storage | Persistent Disk / Hyperdisk | Filestore |
| Azure | Blob Storage | Managed Disks | Azure Files |
Durability, availability, and what the difference means in practice
Durability is the probability that stored data is never lost. Object storage is commonly quoted at "eleven nines" (99.999999999%), achieved by replicating every object across multiple devices. S3, Cloud Storage, and Azure Blob Storage's locally redundant storage (LRS) all publish this figure. Availability is a separate number: the probability the data can be reached right now. Data can be durable and still unreachable during a service disruption. Durability also doesn't cover mistakes: a deleted or overwritten object is gone from every replica, which is what versioning and backups are for.
Lifecycle policies and archival tiers
Object storage in particular supports lifecycle policies: rules that automatically move objects between storage tiers, or delete them, based on age or access pattern. A typical policy moves an object from a standard tier to a cheaper infrequent-access tier after 30 days, then to a much cheaper archival tier (retrieval takes minutes to hours, but storage cost drops sharply) after 90, then deletes it entirely after some retention period if that's appropriate for the data.
Replication: regional vs. multi-region
S3 Standard stores objects across at least three availability zones and Cloud Storage regional buckets across at least two, enough to survive a single datacenter failure. S3 One Zone-IA keeps one zone by design. Azure leaves it to the redundancy option you choose: locally redundant storage (LRS) keeps every copy in one datacenter, so a fire or flood there can destroy all replicas, while zone-redundant storage (ZRS) spans availability zones, and geo-redundant (GRS) and geo-zone-redundant (GZRS) storage add a second region. Check which option an existing setup uses.
Multi-region replication copies data into a second region, protecting against the loss of a whole region and often reducing read latency near that region. It costs more in storage and data transfer, so adopt it when a requirement calls for it: disaster recovery objectives, data residency, or latency for a global user base.