Catalogue › Data Governance › ai-data-quality

AI data quality

Sets data quality requirements for structured and operational data used by AI - accuracy, completeness, currency, consistency, uniqueness and meaning - with thresholds by use and risk tier, ways to measure them, owners and remediation, so agents and analytics do not confidently repeat bad data. Use when an agent reads business records, when Copilot analyses tables or semantic models, or when AI answers trace back to wrong data.

Department

Data Governance

Lifecycle

Envision, Deliver, Assure, Improve

Maturity

builder, scaler, ai-enterprise

Risk tiers

1, 2, 3, 4

Platforms

fabric, power-platform, copilot-studio, ai-foundry, m365-copilot, azure

Editions

starter, scale, enterprise

Reviewers

evaluation-quality

Status

draft

Copilot Cowork

Data Governance plugin for Copilot Cowork, or this skill on its own: Download SKILL.md

Licence

CC BY 4.0, by Simon Foote

Purpose

AI makes data quality problems louder. A report with a duplicated customer is an annoyance; an agent that confidently tells a customer the wrong balance because it found the duplicate first is an incident. This skill sets data quality requirements for the structured and operational data AI uses, proportionate to the use and risk tier, with ways to measure them, owners and a remediation route. It goes wrong when quality is judged by whether the data looks fine in a demo, or when requirements are set so high that no real dataset could pass.

When to use

• An agent or flow reads business records from Dataverse, a database, a list or a business system.
• Copilot or a data agent will analyse tables, spreadsheets or semantic models for decisions.
• AI answers have been traced back to wrong, missing or out-of-date data.
• A data product is being prepared for AI consumption and needs quality expectations.
• Data will be used to build evaluation sets or to tune a model.

When not to use

• Content quality standards for documents and knowledge - use knowledge-management:content-quality.
• Building evaluation sets of questions and expected answers - use evaluation-quality:golden-dataset.
• Judging whether a whole data product is ready for AI - use data-governance:data-product-readiness.
• Bias and unfairness arising from data - use responsible-ai:fairness-review.

Questions to ask first

1. What will AI do with the data - answer questions, summarise, analyse, act, or tune a model?
2. What happens if a value is wrong, missing or stale - who is affected and how badly?
3. Which fields actually matter to the AI use, and which are irrelevant?
4. What is known about the data's current quality, and how was it measured?
5. Who owns fixing it, at source, when it falls short?

Required inputs

• The AI use description and risk tier.
• The data assets involved, from data-governance:ai-data-inventory.
• Any existing quality measures, profiling results or known issues.
• The data owner's availability to agree requirements and fixes.

Where current quality is unknown, profile a sample before setting requirements. Never certify data as fit for AI without measurement, and never set a threshold the owner has not agreed.

Decision framework

Set requirements per critical field, by quality dimension and the consequence of error.

Practitioner approach

Start from the use, not the dataset. A field that is irrelevant to the AI use does not need to be perfect, and trying to fix all data before any AI use is how programmes stall. Ask the business owner what the AI will do with the data and which fields drive its answers or actions. Those are the critical fields; everything else is noted and left.

Profile before you judge. Take a sample and measure: how many critical fields are empty, how old the latest updates are, how many records look like duplicates, how many values fall outside expected ranges. Profiling tools in the data platform help, but a careful look at a few hundred records with the owner often reveals more than a dashboard. Record the results as the baseline.

Pay special attention to meaning. Language models interpret fields by their names and descriptions. A column called "status" with codes 1 to 7, or "amount" without currency, invites confident misreading. Clear descriptions, units, code meanings and synonyms are a quality dimension for AI in their own right, and they are cheap to fix.

Agree requirements with the owner using the tier table. For tier 2 and above, write thresholds for critical fields and how they will be measured. Where data cannot meet them in time, decide how the design will compensate: show data age to users, withhold stale records, require a person to confirm before acting, or narrow the use to fields that are reliable.

Fix at source. Correcting data inside an AI solution, through instructions or a cleaned copy, hides the problem and creates a second version of the truth. Route fixes to the owner and the system of record, and track them.

Monitor after release. Quality drifts, especially when upstream processes change. Agree what is measured, how often, and who is told when a threshold is breached. Feed AI failures that trace back to data into the same route.

Outputs

• Data quality requirements for the AI use: critical fields, dimensions, thresholds by tier, measurement method and frequency, owner.
• A baseline quality profile and a verdict (fit, fit with conditions, not fit) with conditions recorded.
• A remediation plan for gaps, owned by the data owner, with dates.
• Field descriptions and code meanings improved where meaning was the gap.

Microsoft platform considerations

Data used by AI may live in Dataverse, Fabric, Azure data services or business systems reached through connectors, each with its own profiling and monitoring options. Microsoft Purview and Fabric offer data quality capabilities whose scope changes; confirm current features in the relevant source records before designing measurement around them.

Adoption considerations

Users lose trust quickly when AI repeats data they know is wrong. Showing data age and source in answers, and giving users a route to report bad data, turns them into part of the quality loop.

Knowledge considerations

Documents and knowledge content have their own quality standards in knowledge-management:content-quality. Where a solution uses both, agree which source wins when they conflict.

Security and privacy considerations

Accuracy of personal data is a data protection principle, not only a quality preference. Where AI uses personal data, inaccurate records are also a privacy finding; route them to security-privacy:privacy-assessment.

Responsible AI considerations

Missing or inaccurate data is often uneven across groups, and AI can amplify that. Pass patterns of uneven quality to responsible-ai:fairness-review.

Evaluation requirements

Evaluation sets should include cases built on known data problems, such as duplicates, missing values and stale records, so the solution's handling of bad data is tested. Agree these with the evaluation-quality reviewers.

Value measures

Track quality of critical fields against thresholds, AI failures traced to data, and time to fix at source. The data owner owns field-level measures; the data governance lead owns the portfolio view, baselined at first profiling.

Scale notes

Starter

Profile the critical fields for each AI use with the owner, document known issues, and design around them. Avoid tier 3 uses on data that has never been measured.

Scale

Use a standard requirements template by tier, automate profiling for data feeding tier 2+ solutions, and review quality in the operations cadence.

Enterprise

Integrate AI quality requirements into enterprise data quality management and data contracts, and monitor critical fields continuously for tier 3 and 4 uses.

Hand-offs

• To the data owner with the remediation plan.
• To data-governance:data-product-readiness with the quality verdict.
• To evaluation-quality:golden-dataset with bad-data test cases.
• To ai-operations:ai-solution-monitoring with thresholds to monitor.

Reviewer gates

From risk tier 2, evaluation-quality checks that quality requirements exist for critical fields and that evaluation covers bad-data cases. Below tier 2 this is advisory guidance for the owner.

Failure modes

• All data must be perfect before any AI use, so nothing ships.
• Data looks fine in a demo, and duplicates or stale records surface with real users.
• Fixes are made in a cleaned copy for the agent, creating a second version of the truth.
• Field meaning is ignored, so the model misreads codes and units with confidence.

Sources

references/sources.md in this skill lists the source records behind this skill, what each is authoritative for, how volatile it is and what you may do with it. Product availability, licensing, limits and preview status change often, so check the current source before stating them, and cite what you used.

Never

• Certify data as fit for AI without measuring it.
• Fix data inside the AI solution instead of at source.
• Set quality thresholds without the data owner's agreement.
• Treat field descriptions and code meanings as optional for AI use.