FinOps: Cost per Query, Cost per Pipeline

The bill is a query you can write

Level 6 taught you the serverless cost model as a fact: a query engine like Athena charges for the bytes it reads, not the rows it returns. FinOps is what happens when that fact becomes a weekly habit. Somebody asks "why did the platform bill jump 40 percent", and the useful answer is not a shrug, it is a result set. Every managed engine writes its own usage metadata, and every usage table has the same two things in it: a unit of work, and a meter reading.

The two meters that cover most of a data platform:

• Bytes scanned. Athena bills $5.00 per TB scanned, with a 10 MB minimum per query. The minimum is not trivia. It means a query against a two-megabyte dimension table still costs the same as a ten-megabyte one, so a tile that refreshes every thirty seconds against a tiny table is billed thousands of times for bytes it never read.
• DPU-hours. A Glue ETL job bills $0.44 per DPU-hour (a DPU is 4 vCPU and 16 GB), with a one-minute minimum per run. A job that finishes in eleven seconds still bills a minute, which is why "one Glue job per table" looks cheap in the code review and expensive on the invoice.

Cost per unit of work is then plain SQL: take the meter reading, apply the floor, apply the rate.

Table

Four meters, one query shape: reading times rate, grouped by whatever you want to attribute the money to.

meter	billed unit	the audit query	the usual leak
bytes scanned	MAX(bytes_scanned, 10 MB) at $5/TB	cost per query, per user, per table	one dashboard rescanning raw files
DPU-hours	DPU-hours at $0.44, 1-minute floor	cost per pipeline per month	many tiny jobs paying the minimum
storage	GB-month per storage class	spend by prefix and class	no lifecycle rule, nothing ages out
requests	GET/PUT per 1,000	requests per partition	millions of small files

The leak hides from the ranking

Here is the part that catches people. Rank your queries by cost and you get the ten biggest single scans, which are almost always somebody's one-off backfill or an analyst exploring. Those are visible, defensible, and usually not the problem.

The problem is the query nobody notices, run over and over. AWS's own documented case is a team whose Athena spend doubled in a quarter, with about 70 percent of it repeated scans of the same data. No single run looked alarming. The fingerprint that finds it is a hash of the normalized SQL text, carried in this lesson's seed as query_hash: identical text means an identical hash, so the same query run four hundred times collapses into one row. Group by the hash instead of by the query and the shape of the bill changes completely.

Once you have the hash you also have the fix menu, in order of effort: cache or materialize the dashboard's result so it reads a small table instead of the raw one; convert and partition the source so each run reads a fraction of the bytes (the next lesson prices that); or set a workgroup bytes-scanned limit, which is the platform's built-in brake and cancels a query that goes past a byte budget, so you pay only for what it scanned before the cancel instead of for the full runaway scan. Be precise about that last one in an interview: the limit caps the blast radius, it does not zero it. A cancelled query is still billed for the bytes it read on the way to being cancelled, exactly like one you cancel by hand.

Attribution is the deliverable, not the number

A FinOps finding that stops at "we spent $12,400" changes nothing. The finding that changes something names an owner: this hash, run by this service account, against this table, is 69 percent of the bill. That is why every query in this lesson groups by something a human is responsible for.

Common mistake: billing on bytes_scanned directly and forgetting the minimum. It makes no difference to your top spenders and a very large difference to any account whose workload is thousands of tiny queries, which is exactly the account that shows up asking why a "free" dashboard has a bill.

Interview nuance: when you are asked "how would you reduce our warehouse costs", the answer that lands is not a list of tips, it is a method: attribute first (per query, per hash, per user, per pipeline), then fix the biggest attributed line, then put a guardrail on it so it cannot come back. Naming the repeated-scan pattern, and grouping the history by a hash of the normalized query text as the way you would find it, is the detail that separates you from a candidate reciting "use Parquet".

On a real platform this differs. Here you query one small query_history table in SQLite, and the hash arrives pre-computed. Not every platform hands you one. Snowflake does, as QUERY_HASH and QUERY_PARAMETERIZED_HASH on its query history views. Athena, BigQuery and Redshift do not: their history gives you the query text, the bytes scanned and the status, so you normalize the text and hash it yourself, or simply group by the text. That is the only step this seed takes for you. Otherwise the same reasoning runs against information_schema-style query history, the Cost and Usage Report, or CloudWatch job metrics. On BigQuery the meter is on-demand bytes billed per job in INFORMATION_SCHEMA.JOBS; on Microsoft Fabric it is capacity units in the metrics app. The rate and the column names move, the query shape does not.