Shrink the failing input

Level 5: Level 5: Judging Code You Did Not Writemedium22 mindebuggingminimal reproductionbisectionverification

A bug report with a 400 row file is not a diagnosis. Cut the input down until removing anything else makes the failure disappear, and the answer is usually visible.

"It fails on the production file"

That sentence is where most debugging sessions start and it contains almost no information. The file has forty thousand rows, the failure is somewhere in the interaction between them, and reading the function again for the fourth time is not going to reveal which.

The move is not to read harder. It is to make the input smaller until it is small enough to understand.

A minimal reproduction is an input that still fails, where removing any part of it makes the failure go away. Getting there is mechanical, it needs no insight, and it usually hands you the diagnosis for free, because by the end the input has been reduced to precisely the thing that matters.

Bisect when the input is a sequence

Halve it. Test the half. Whichever half still fails is your new input. Repeat.

rows = load("production.csv")        # 40000 rows, fails
first_half = rows[:20000]            # fails    -> keep going here
first_quarter = rows[:10000]         # passes   -> the trigger is in 10000..20000

Sixteen halvings take forty thousand rows down to one. Sixteen. That is the entire reason this technique is worth learning: the cost is logarithmic in the size of the thing you are afraid of.

Check yourself

A 40000 row file makes an import crash. Roughly how many runs does halving need to reach a single triggering row?

About 16About 200, the square root of the row countAbout 40000, since every row has to be triedIt depends entirely on where the bad row sits

Prefixes when the failure depends on history

Halving assumes any subset can be tested on its own. Plenty of failures do not work that way, because the function carries state forward and the bug depends on what came before.

def apply_ops(ops):
    balance = 100
    for op in ops:
        balance += op        # the contract says an op that would overdraw is skipped
    return balance

No single op is bad here. -80 is fine when the balance is 200 and wrong when the balance is 50. So the unit to shrink is the prefix: try ops[:1], ops[:2], ops[:3], and stop at the first length that misbehaves. The last operation in that prefix is the one that tipped it over, and the prefix before it is the state that made it possible.

def shortest_failing_length(ops):
    for n in range(1, len(ops) + 1):
        if apply_ops(ops[:n]) != correct_apply(ops[:n]):
            return n
    return -1

Check yourself

For apply_ops above with ops = [10, 20, -500, 5], which shrinking strategy finds the trigger?

Test each operation on its own, since one of them must be the bad oneTest growing prefixes and stop at the first length that misbehavesRemove the largest value and see whether the failure goes awaySort the operations first, so the shape of the input is normalised

Shrink the value, not just the length

Once the input is short, keep going on the contents. Replace long strings with short ones, big numbers with small ones, real names with "a". Every simplification that preserves the failure removes a thing you would otherwise have to think about.

A good stopping point looks like this:

assert apply_ops([-200]) == 100    # fails: returns -100

One operation, round numbers, and the bug is now impossible to misread: the function applied an operation that should have been skipped. Compare that to "the balance is wrong after importing the March file".

Check yourself

Sort each reproduction by whether it is finished shrinking or still has work left.

assert average([2, None]) == 2.0

The nightly job fails when run against the staging database

assert parse('2026-13-01') raises ValueError

It crashes on customers.csv, attached, 12MB

It returns the wrong total for orders where the discount is applied twice and the customer is in the EU and the currency is not USD

Keep the shrunk input

The minimal reproduction is not scaffolding you throw away when the fix lands. It is the regression test, already written, in the form that is easiest to read. Copy it into the suite and name it after what it proves.

Interview nuance: given a failing test and a large input, the strongest opening move is out loud: "before I read the function, let me cut this input down." It shows you know that debugging is a search problem rather than a reading problem, and it converts a vague situation into a small one within a minute. Candidates who start by reading the function line by line are still reading it when the time runs out.

Check yourself

You have shrunk a 40000 row failure down to two rows. Removing either row makes it pass. What have you got?

Two suspicious rows, which you now need to inspect for bad dataA minimal reproduction, which points at the interaction between them as the bugProof that the file was corruptAn input too small to be representative, so you should scale it back up

Worked example (Python)

def apply_ops(ops):
    balance = 100
    for op in ops:
        balance += op      # the contract says an overdrawing op is skipped
    return balance


for n in range(1, 5):
    prefix = [10, 20, -500, 5][:n]
    print(prefix, "->", apply_ops(prefix))

Apply

Your turn

The task this lesson builds to.

Write a function shortest_failing_length(ops) that returns the length of the shortest prefix of ops on which apply_ops disagrees with its contract, or -1 when no prefix does.

The contract: apply_ops(ops) starts from a balance of 100 and applies each operation in order, skipping any operation that would take the balance below zero. It returns the final balance.

The generated version in the starter never skips anything. No single operation is wrong on its own, so shrink by prefix rather than by element. Keep shortest_failing_length as the last function in the file.

3 hints and 4 automated checks are waiting in the workspace.

Practice

Make it stick

A second problem on the same idea, so it survives past today.

Your latency dashboard went blank overnight and the on-call engineer pasted a list of two thousand recorded response times into the ticket with the note "it crashes on this". You need something a reviewer can read in one line before you can even start on the fix.

Write a function smallest_failing_prefix(times) that returns the shortest prefix of times (as a list, with at least one element) on which average_response disagrees with its contract, or an empty list when no prefix does.

The contract: average_response(times) returns the mean of the recorded times, ignoring any entry recorded as None because that request never completed, and returns 0.0 when there is nothing left to average. Keep smallest_failing_prefix as the last function in the file.

3 hints and 4 automated checks are waiting in the workspace.