AI data governance is ordinary data governance with higher stakes, because data that trains or feeds an AI system can shape its behaviour and is hard to take back out. Four questions carry most of the weight: where the data came from, whether people agreed to its use, whether it is good enough and fair, and how long it is kept. Regulators have acted on each.
01What changes when data feeds an AI system?
Normal data governance asks who owns data, who may use it and when it is deleted. AI adds a twist. Data used for training can end up shaping a model, and once a model is trained, pulling one person's data back out of it is not simple. So the decisions about what goes in matter more at the start.
The EU AI Act spells this out for high-risk systems. Its Article 10 asks providers to apply data governance practices to training, validation and testing data (AI Act, Article 10). Those duties apply from 2 December 2027 for the listed high-risk uses, as of 24 October 2026. This page is general information, not legal advice; ask a lawyer or your data protection regulator about your case.
02Where did the data come from, and did people agree?
Provenance means the data's origin story. Article 10 asks for records of where data came from, how it was collected and what it was first collected for, especially for personal data. A well-known research proposal, Datasheets for Datasets, suggests every dataset ships with a short document covering its motivation, contents, collection process and recommended uses.
Consent is harder. A 2024 audit of 14,000 web domains behind common AI training collections found a growing number of AI-specific restrictions in site terms, and a mismatch between what sites say in their terms and what their crawler rules express. The authors judge the web's signalling tools were not built for this use (Longpre et al.).
Regulators are still working out the legal basis for training on personal data. The European Data Protection Board gave an opinion on AI models in December 2024 and opened consultation on web scraping for generative AI in July 2026 (EDPB Opinion 28/2024).
03Is the data good enough, and is it needed?
Article 10 asks that data be relevant, representative for the people affected, as free of errors as possible and checked for bias that could harm health, safety or rights. It also asks providers to name gaps in the data and plan how to fix them.
The other half of quality is necessity. In December 2021 the Dutch data protection authority fined the Dutch Tax Administration EUR 2.75 million. It found nationality data was kept and used as an indicator in an automated system that flagged childcare benefit applications as risky, though the data was not needed for that purpose (Autoriteit Persoonsgegevens). Collecting less can be the safest quality control.
04How long should data be kept, and can it be deleted?
In May 2023 the FTC and Department of Justice alleged that Amazon kept children's Alexa voice recordings indefinitely unless a parent asked for deletion, that some transcripts stayed in databases after deletion requests, and that the recordings were used to train its speech algorithm. Amazon agreed to a USD 25 million civil penalty. The proposed order would bar using data covered by deletion requests to train or improve algorithms (FTC).
Rules can also reach the model. The FTC's proposed Rite Aid order, announced in December 2023, would have the company delete images and algorithms built from them (FTC). How far deletion duties can reach into trained models is an open question.
Regulator findings can be contested. Italy's privacy regulator fined OpenAI EUR 15 million in December 2024, but its own page now says the decision was temporarily removed after a Rome court accepted OpenAI's appeal in March 2026 (Garante).
05What five questions should you ask before data goes into an AI tool?
This is our own plain checklist, a starting point and not a compliance test.
- Where did this data come from, and what was it collected for?
- Did the people it describes agree, or does a clear legal basis exist?
- Is it accurate, current and representative of the people the tool will affect?
- Do we need all of it, or only part?
- How long will it be kept, who can ask for deletion, and can the vendor delete it from systems that learned from it?
?What is not known
- Regulators have not settled when training on personal data has a valid legal basis; the EDPB consultation on web scraping was still open when we read its page.
- How far a deletion request can reach into a model that has already been trained is unresolved.
- The EU AI Act data duties for high-risk systems have moved to December 2027 and the text may change again.
QQuestions people ask
Is AI data governance a separate job from data governance?
It can be the same people with new questions added, such as whether data is used for training, how long a vendor keeps it, and whether it has been checked for bias.
Does using a chatbot at work count as a data governance issue?
Yes, if people paste in customer, staff or confidential data. Your written rule should say what may never be pasted in and which tool settings are required.
Last checked October 24, 2026. Rules and settings change, so open the source to see the current version.
- AI Act, Article 10: Data and data governance Future of Life Institute, AI Act Explorer Non-official copy of the amended article, used for the data governance practices and quality criteria.
- Datasheets for Datasets arXiv, published in Communications of the ACM, 2021 Research paper proposing a standard document for each dataset; used for the idea of documenting dataset origin and contents.
- Consent in Crisis: The Rapid Decline of the AI Data Commons arXiv preprint (Longpre and 48 co-authors) Audit of 14,000 web domains used for what site owners signal about AI training use.
- Opinion 28/2024 on AI models and personal data European Data Protection Board Official page used for the opinion date and for the July 2026 web scraping guidelines consultation listed beside it.
- Tax Administration fined for discriminatory and unlawful data processing Autoriteit Persoonsgegevens (Dutch DPA) Regulator's announcement of the fine of 7 December 2021.
- FTC and DOJ charge Amazon over kids' Alexa voice recordings US Federal Trade Commission Regulator's description of the allegations, the penalty and the proposed order.
- Rite Aid banned from using AI facial recognition US Federal Trade Commission Used for the proposed order's requirement to delete images and algorithms built from them.
- Garante press release on OpenAI Garante per la protezione dei dati personali (Italy) Regulator's December 2024 announcement and its note on the later court judgment.
General information, not legal, medical, financial or investment advice. We are not affiliated with any company, agency or standards body named here. See our editorial policy and corrections, or tell us about a mistake at [email protected].