Generators, yield & iterators

Why laziness matters

When you scan a 10 GB log file to count errors, you do not want the whole file sitting in memory at once. A generator lets you process one line at a time, so memory stays flat no matter how large the input grows. That is why streaming pipelines, database cursors, and csv.reader all hand you values lazily instead of returning a giant list. Building the full list first is often the difference between a job that finishes and a job that gets killed for using too much memory.

A generator is a paused function

Table

list(countdown(3)) drives the generator: each pull runs the body to the next yield, hands back one value, then freezes with n intact until the next pull. The 4th pull finds no more yields and raises StopIteration.

pull	body runs to	yields	n now
next() #1	yield n	3	3
next() #2	n -= 1; yield n	2	2
next() #3	n -= 1; yield n	1	1
next() #4	loop ends, StopIteration	NULL	0

A generator function uses yield instead of return. Calling it does not run the body. It hands you a generator object. Each time you ask for a value (via for, next, sum, and friends), the body runs until it hits a yield, hands back that value, and freezes right there with every local variable intact. The next request resumes on the line after the yield.

def countdown(n):
    while n > 0:
        yield n          # hand back one value, then pause here
        n -= 1

print(list(countdown(3)))   # [3, 2, 1]

Nothing is computed until you iterate. list(...) drives the generator to exhaustion; a for loop pulls exactly one value per pass.

Generator expressions

Same laziness, compact syntax: a comprehension with parentheses instead of brackets. No intermediate list is built.

print(sum(n * n for n in range(1, 5)))   # 1 + 4 + 9 + 16 = 30

range(1, 5) yields 1, 2, 3, 4, so this sums the first four squares. For the Apply exercise you will want range(1, n + 1) so that n itself is included, and n = 0 produces an empty range whose sum is 0.

Stop at the first match with next

next(gen) pulls one value. next(gen, default) returns default instead of raising when the generator is empty. Paired with a filtered generator expression, it stops at the first hit and never touches the rest.

nums = [0, 0, 7, 3]
print(next((x for x in nums if x), None))   # 7

That is exactly the shape for finding the first truthy value: the if x filter keeps only truthy items, and next(..., None) supplies the fallback when nothing qualifies.

Pitfall: a generator is single-use

Iterating a generator consumes it. There is no rewind.

g = (n * n for n in range(1, 5))
print(sum(g))   # 30
print(sum(g))   # 0   <- already exhausted, nothing left to yield

The second sum sees an empty generator, not a fresh one. Also, next(gen) with no default raises StopIteration on an empty generator, so always pass a default when "not found" is a real outcome.

Interview nuance: a generator is an iterator. It implements __iter__ (returning itself) and __next__, holds one value at a time, and runs in O(1) extra memory no matter how many values it produces. A list comprehension is O(n) memory and re-iterable; a generator is O(1) memory and single-pass. Interviewers probe that trade-off directly: reach for a generator when you consume the sequence once and streaming saves memory, and for a list when you need indexing, len, or more than one pass.