Object vs Block vs File Storage, and Why the Lake Lives on S3
The three storage shapes every cloud offers (S3, EBS, EFS), the immutable-blob-over-HTTP object model, eleven-nines durability, and why a data lake is files in a bucket.
Three ways to store bytes
Before you can reason about a data lake you need the three storage shapes every cloud offers, because interviewers ask for them by name and juniors mix them up constantly.
- Block storage is a raw virtual disk. You attach it to one machine, format a filesystem on it, and read or write any block in place, exactly like the SSD in a laptop. On AWS this is Amazon EBS. It is what a boot disk and a transactional database's files sit on, because a database needs to update a few kilobytes in the middle of a file, in place, with sub-millisecond latency.
- File storage is a shared folder many machines mount over a network protocol (NFS). On AWS this is Amazon EFS. Reach for it when many servers must read and write the same files at once.
- Object storage is a giant key-value store of whole files, reached over HTTP. On AWS this is Amazon S3, and it is where data lakes live.
| dimension | block (EBS) | object (S3) | file (EFS) |
|---|---|---|---|
| access unit | blocks of a raw disk | whole object + key | files in a folder |
| addressed by | device + offset | key over HTTP | path over NFS |
| mutability | edit any block in place | replace the whole object | edit files in place |
| shared by | one instance | any number of readers | many instances at once |
| use it for | boot disk, a database's files | data lake, backups, logs | shared files across machines |
Why the lake lives on object storage
An object is a file plus its metadata, addressed by a key inside a bucket, for example the key raw/events/2026-01-02/part-0.parquet. You reach it with an HTTP GET or PUT, and every object has a URL. Three properties make this the natural home for a data lake:
- It scales without limit and costs little. A bucket holds any number of objects, and a single object can be up to 5 TB. You pay only for what you store, with no capacity to provision.
- It is extremely durable. S3 is designed to provide eleven nines of durability (99.999999999%) by storing every object redundantly across at least three Availability Zones. That is a design target for not losing data, and it is a different number from availability (99.99%, the chance you can reach it at a given moment). Mixing up durability and availability is the single most common junior mistake here.
- Any engine can read the files. Because objects are just files behind an HTTP key, Athena, Spark, and a warehouse can all read the same bucket. Storage is decoupled from compute.
The trade is that object storage is not a disk. You cannot edit an object in place: a write replaces the whole object (or, with versioning on, adds a new version). There is no "change these 8 bytes." That is fine for a lake, where you write files once and read them many times, and it is exactly why a live transactional database wants EBS, not S3.
S3 is also strongly read-after-write consistent. Since December 2020, a read that follows a successful write (a new object or an overwrite) always returns the new bytes, in every Region, at no extra cost. If you hear "S3 is eventually consistent," that is the pre-2021 model and it is obsolete, so do not say it in an interview (only bucket-level configuration changes and the S3 Inventory snapshot are still eventually consistent). One more thing a junior should be able to say out loud: access is not open. Who can read or write a bucket is governed by IAM and bucket policies, S3 encrypts objects at rest by default, and columns holding personal data (PII) need extra care. You do not manage that in SQL, but the bucket is not public by default.
One more property worth stating plainly: S3's namespace is flat. The slashes in a key only look like folders. raw/events/2026-01-02/ is a prefix of the key, not a real directory, and console "folders" are a convenience the UI draws. This matters in the partitioning module, where a layout like dt=2026-01-02/ is just a key prefix that a query engine filters on.
Common mistake: treating S3 like a filesystem. You cannot edit an object in place or append to it, and there are no real directories, only key prefixes, so code that expects to open and rewrite a file in place belongs on a block volume, not the lake.
Interview nuance: "object versus block versus file, give an AWS example of each" is a near-guaranteed junior question. Answer with the access unit and mutability: block is a raw disk you format (EBS), file is a shared mount (EFS), object is whole files over HTTP that you replace rather than edit (S3), and the lake uses object storage for cheap, unlimited, engine-agnostic capacity.
On a real platform this differs. Real S3 inventory comes from an S3 Inventory report (a daily or weekly file listing every object with its size, storage class, and last-modified date) that you query in Athena. The
s3_inventorytable you query here is a small stand-in with the same columns, so the query you write is the query you would run for real.
CREATE TABLE s3_inventory (
object_key TEXT,
size_bytes INTEGER,
storage_class TEXT,
last_access_days INTEGER -- days since the object was last read
);
INSERT INTO s3_inventory (object_key, size_bytes, storage_class, last_access_days) VALUES
('raw/events/2026-01-02/part-0.parquet', 300000000000, 'STANDARD', 1),
('raw/events/2025-08-01/part-0.parquet', 300000000000, 'STANDARD', 150),
('raw/logs/2025-11-15/app.log', 400000000000, 'STANDARD', 210),
('curated/daily_active/2026-01-01.parquet', 90000000000, 'STANDARD', 1),
('raw/logs/2025-10-01/app.log', 600000000000, 'STANDARD_IA', 260),
('archive/2024/full.tar', 1000000000000, 'GLACIER_DEEP',800),
('exports/report-2026-01.csv', 50000000000, 'STANDARD', 5);-- The inventory: each object's size in GB and whether it has gone cold.
SELECT object_key,
ROUND(size_bytes / 1000000000.0, 0) AS size_gb,
CASE WHEN last_access_days > 90 THEN 'cold' ELSE 'hot' END AS access_state
FROM s3_inventory
ORDER BY size_bytes DESC;Apply
Your turn
The task this lesson builds to.
Write a query that returns the total bytes stored under each top-level prefix, as (prefix, total_bytes), largest first, over s3_inventory(object_key, size_bytes, storage_class, last_access_days).
The top-level prefix is the part of object_key before the first slash (for raw/events/... it is raw). Group by that prefix and sum size_bytes. Alias the columns exactly prefix and total_bytes, most bytes first.
3 hints and 1 automated check are waiting in the workspace.
Practice
Make it stick
A second problem on the same idea, plus 4 bonus drills.
Write a query that returns every object not accessed in the last 90 days, as (object_key, size_bytes, last_access_days), largest first, over the same s3_inventory table.
These are the cold objects a lifecycle policy would move to a cheaper storage class. Keep only rows where last_access_days is greater than 90, ordered by size_bytes descending.
2 hints and 1 automated check are waiting in the workspace.