PII Governance, DSAR/Erasure & Privacy Engineering

Erasure is hard because PII is never in one place

GDPR gives users the right to see their data and the right to be forgotten. Honoring the second one, erasure within 30 days across every store, is one of the hardest data-engineering problems most companies quietly fail at, because a user's PII is never in one place.

Step one: know where it lives

You cannot delete what you cannot find. This demands a data inventory and classification: a catalog (DataHub, Amundsen, AWS Glue Data Catalog, or a home-grown registry) that tags every field as PII, sensitive, or non-sensitive, and records which datastore, table, and column holds it. Without this, "delete the user" is a guess. Mature teams enforce classification at write time so new PII columns cannot appear uncatalogued.

Step two: the rights machinery

Beyond access (DSAR) and erasure, GDPR grants rectification (fix wrong data), portability (export in a machine-readable format), and consent withdrawal. Model these as an orchestrated workflow keyed on a stable user_id, with a queue that fans a request out to every system of record and tracks completion, because a 30-day legal deadline needs a status you can audit, not an email thread.

Step three: delete every copy (the hard part)

A single user lives in the primary DB, read replicas, Redis caches, an Elasticsearch/OpenSearch index, the analytics lake (S3/Parquet), message queues, application logs, and third-party processors (Stripe, Segment, your email provider). Erasure has to reach all of them. Live stores you delete directly. Search indexes you re-index or delete by query. Third parties you call their deletion API and record the confirmation.

Backups are the killer. You cannot surgically edit a Postgres snapshot from three weeks ago, and you should not (immutable backups are a ransomware defense). The answer is crypto-shredding: encrypt each user's data with a per-user data key, store those keys in a KMS, and to "erase" the user, destroy their key. The ciphertext still sits in old backups but is now unrecoverable noise, which regulators accept as effective erasure. This is the single most important pattern in this lesson.

Erasure request(user_id)
   -> live DBs / replicas: DELETE rows
   -> caches (Redis): evict keys
   -> search (OpenSearch): delete by query
   -> lake (S3/Parquet): tombstone + compaction rewrite
   -> third parties: call deletion API, store receipt
   -> backups: DESTROY per-user KMS key (crypto-shred)

Retention conflicts are real: tax law may require keeping transaction records for 7 years even after an erasure request. You cannot honor both blindly, so policy is per-field. You erase the marketing profile and contact info while retaining the legally-mandated financial record (often pseudonymized), and you document the lawful basis for what you keep.

Interview nuance: know the three de-identification tiers precisely. Anonymization is irreversible: strip identifiers so no one can re-link (truly anonymized data leaves GDPR scope). Pseudonymization replaces identifiers with a reversible token, key held separately, still personal data under GDPR. Tokenization is a form of pseudonymization for a specific field. For sharing analytics safely, add k-anonymity (each row indistinguishable from at least k-1 others) or differential privacy (inject calibrated noise so no individual's presence changes an aggregate). Aggregates alone are not safe: a single-row group re-identifies instantly.

Recap: catalog every copy first, orchestrate rights on a stable user_id, delete across all live stores plus third parties, crypto-shred to handle backups, resolve retention conflicts per-field, and pick anonymization vs pseudonymization deliberately with k-anonymity or differential privacy for shared analytics.