Skip to main content

Context managers, JSON & CSV

Level 2: Level 2: Idiomsmedium12 mincontext-managersjsoncsvparsing

Read structured data with the with-statement, json, and csv.

Reading structured data safely

Almost every backend or pipeline job starts the same way: pull in text from a file or an API, turn it into Python objects, and do work. Two things go wrong constantly. First, a file handle gets leaked because someone forgot to close it, and under load the process runs out of descriptors and crashes. Second, the parse is naive, so a comma inside a quoted field or a number that arrives as text silently corrupts every downstream row. The standard library fixes both, and interviewers expect you to reach for it instead of hand-rolling.

Context managers guarantee cleanup

with opens a resource and closes it when the block exits, even if an exception is raised partway through:

with open("data.txt") as fh:
    text = fh.read()
# fh is closed here, exception or not

open returns a context manager: the with statement calls its setup on entry and its cleanup (fh.close()) on exit. This is the idiom for anything that needs releasing (files, sockets, locks, DB connections). The raw data in these exercises is already handed to you as a string, so you go straight to parsing.

The second argument to open is the mode, and picking the wrong one is how files get erased:

Check yourself
A 2 GB log file exists. You run open(path, 'w') meaning to add a line to it, and the process is killed before you write anything. What is in the file afterwards?
Table
The w row is the one to remember: opening for write destroys the contents before you have written anything, so a crash straight after open still loses the file. Use a when you mean to add, and x when creating something that must not already exist.
ModeIf the file existsIf it does notWrites go
r (the default)opened for readingFileNotFoundErrornowhere; reading only
wTRUNCATED to empty immediatelycreatedfrom the start
aopened, contents keptcreatedalways appended to the end
xFileExistsErrorcreatedfrom the start
r+opened for read and writeFileNotFoundErrorfrom the start, overwriting in place
The w row is the one to remember: opening for write destroys the contents before you have written anything, so a crash straight after open still loses the file. Use a when you mean to add, and x when creating something that must not already exist.

JSON: text to Python objects and back

json.loads parses a JSON string into Python values; json.dumps serializes back to a string. The type mapping is fixed: JSON object to dict, array to list, string to str, number to int or float, true/false to bool, null to None.

import json

data = json.loads('{"name": "Ada", "age": 30}')
data["name"]   # "Ada" (a str)
data["age"]    # 30    (an int, not "30")

Note data["age"] is a real integer, so you can do arithmetic on it directly. Use json.load/json.dump (no s) when the source or target is a file object rather than a string.

CSV: rows of strings

csv.reader yields one list per row and handles quoted fields and embedded commas correctly. It reads from any line-yielding iterable, so wrap a string in io.StringIO to treat it like a file:

import csv, io

rows = list(csv.reader(io.StringIO("a,b\nc,d")))
# [["a", "b"], ["c", "d"]]
Check yourself
csv.reader parses the row Ada,30 into a list. You take the second value and add 1 to it. What happens?

Every CSV value comes back as a str. There is no type inference: the number 30 in a CSV cell arrives as "30", and you must call int() yourself.

Pitfalls

  • Do not split(",") a CSV line. For '"a,b",c', str.split(",") returns three broken pieces, while csv.reader correctly returns ["a,b", "c"] because it respects the quotes.
  • loads vs load. Pass a string to json.loads and a file object to json.load. Mixing them raises TypeError or AttributeError.
  • When reading a real CSV file, open it with newline="" so csv handles line endings itself: open(path, newline="").
Check yourself
point = (1, 2). What is json.loads(json.dumps(point)) == point?

Interview nuance: JSON round-tripping is lossy for Python types. json.dumps((1, 2)) produces the array "[1, 2]", and json.loads of that gives back the list [1, 2], not the original tuple. JSON has no concept of tuples, sets, or non-string dict keys, so tuples become lists, sets are unserializable, and integer keys are coerced to strings. If your code depends on getting the exact Python type back after a serialize-then-parse cycle, it will break.

Worked example (Python)
import json

data = json.loads('{"name": "Ada", "age": 30}')
print(data["name"])   # Ada
print(data["age"])    # 30

Apply

Your turn

The task this lesson builds to.

Implement get_field(raw, field): parse the JSON string raw and return the value stored at field.

For raw = '{"name": "Ada", "age": 30}' and field = "name", return "Ada".

3 hints and 3 automated checks are waiting in the workspace.

Practice

Make it stick

A second problem on the same idea, so it survives past today.

Implement parse_csv(text): parse the CSV string text into a list of rows, where each row is a list of its string values.

For "a,b\nc,d" return [["a", "b"], ["c", "d"]]. Use csv.reader over an io.StringIO.

3 hints and 3 automated checks are waiting in the workspace.