Why Access Control Cannot Enforce Purpose Limitation in Data Pipelines

In May 2022, the FTC and the U.S. Department of Justice fined Twitter $150 million. Twitter collected phone numbers and email addresses from more than 140 million users for account security. Advertisers then used the same data to target ads. The collection purpose was stated to users, but it did not restrict which systems could read those fields.

In most organizations, policy text and consent databases store the allowed purposes of each record. A pipeline that reads the record gets an access decision only. It does not get the allowed purposes.

Requirement: purpose limitation

Purpose limitation means that you use personal data only for the purposes for which you collected it.

GDPR Article 5(1)(b). You collect data for specified purposes. You do not process it further in an incompatible way.
CCPA, Cal. Civ. Code § 1798.100(c). Collection, use, retention, and sharing must be reasonably necessary and proportionate to the purpose of collection.
FTC Act Section 5. A use of data outside what a company told consumers can be a deceptive practice. (The Twitter case is an example)

The requirement applies to each record. A record can be allowed for fraud detection and not allowed for ad personalization. The allowed set changes when the user changes a consent setting.

Access control and data use

Data platforms enforce access control at the table, column, or row level. An access check evaluates the principal and the resource. It does not evaluate the purpose of the job or the allowed purposes of each record. After the read, the record goes through joins, feature transforms, copies, and training sets. These operations do not copy the collection purpose as metadata with the record. A downstream job has no field that shows that a phone-number column came from an account-security flow. Enforcement then depends on code review against the written policy and on audits after the data is used. Neither control runs automatically on each pipeline change before deployment.

Published evidence

Andow et al. (USENIX Security 2020) compared the privacy policies of 13,796 Android apps with the data flows of those apps. Up to 42.4% of the apps disclosed their privacy-sensitive flows incorrectly or did not disclose them.
A Kiteworks survey of 225 security, IT, and risk professionals (December 2025) reports that 78% of organizations cannot validate data before it enters a training pipeline. 77% cannot trace the source of their training data. These figures are self-reported survey answers from a vendor study. They are not independent measurements.

Impact on model training

If a pipeline sends restricted data to a report, you fix the pipeline and delete the output. If a pipeline sends restricted data to a training job, the data becomes part of the model weights. You cannot remove it without retraining.

The FTC has ordered companies to delete models that they trained on improperly obtained data: Everalbum (2021), WW International and Kurbo (2022), and Rite Aid (2023). A training set often joins many source tables, and one model can serve many products. A purpose violation in one source table propagates to each model trained on it and to each product that uses those models.

Minimum enforcement design

Denning published a lattice model of secure information flow in 1976. Information flow control applies that model to programs and operating systems. The requirement here is to apply it to production data platforms and training pipelines. The minimum design has four parts:

Purpose labels on data. Each record or field carries its allowed purposes. Joins, transforms, and copies keep the labels.
Declared purpose on jobs. Each pipeline and training job declares its purpose.
Check at the point of use. The system compares the job purpose with the data labels. It removes or blocks the records that are not allowed.
Pre-deployment testing. You test each change to a pipeline or a policy before production. The test finds violations. It also finds allowed data that the change drops by mistake.

Figure 1 shows parts 1 to 3 in a training pipeline.

Figure 1.

Self-assessment

When a user withdraws consent for a purpose, which systems receive the change, and how fast?
Can you list each training set and model that used a given dataset?
Does a system, and not a person, compare the purpose of each job with the data that it reads?
When a privacy policy changes, how does the change get into code?

If the answer to a question is a manual process or a document, the system does not enforce that control.