Privacy by Design laws will kill your data pipelines

A car is totaled when the cost to repair it exceeds its total value. By that logic, Privacy by Design legislation could soon be totaling data pipelines at some of the most powerful tech companies.

Those pipelines were developed well before the advent of more robust user privacy laws, such as the European Union’s GDPR (2018) and the California Consumer Privacy Act (2020). Their foundational architectures were therefore designed without certain privacy-preserving principals in mind, including k-anonymity and differential privacy.

But the problem extends way beyond trying to layer privacy mechanisms on top of existing algorithms. Data pipelines have become so complex and unwieldy that companies might not even know whether they are complying with regulations. As Meta engineers put it in a leaked internal document: “We do not have an adequate level of control and explainability over how our systems use data, and thus we can’t confidently make controlled policy changes or external commitments.”

(When we asked Meta for comment, a spokesperson referred us to the company’s original response to Motherboard about the leaked document, which said, in part: “The document was never intended to capture all of the processes we have in place to comply with privacy regulations around the world or to fully represent how our data practices and controls work.”)

As governments increasingly embrace Privacy by Design (PbD) legislation, tech companies face a choice: either start from scratch or try to fix data pipelines that are old, extraordinarily complex and already non-compliant. Some computer science researchers say a fresh start is the only way to go. But for tech companies, starting over would require engineers to roll out critical data infrastructure changes without disrupting day-to-day operations — a task that’s easier said than done.

‘Open borders’ won’t cut it

Motherboard published the leaked internal document, written by Meta engineers in 2021, at the end of April. In it, an engineering team recommended data architecture changes that would help Meta comply with a wave of governments embracing the “consent regime,” one of the core principles of PbD. India,…