Catalogue › Microsoft Foundry › foundry-production-readiness

Foundry production readiness

Assembles and tests the evidence that a custom Microsoft Foundry solution is ready for live use - evaluation and safety results, security and identity, deployment and capacity, observability, cost controls, resilience, support, model lifecycle and rollback - and records a go, go with conditions or no-go. Use when a Foundry build is nearing release, when a pilot is about to widen, or when a partner hands over a solution.

Department

Microsoft Foundry

Lifecycle

Assure, Launch

Maturity

builder, scaler, ai-enterprise

Risk tiers

1, 2, 3, 4

Platforms

ai-foundry, azure, entra

Editions

starter, scale, enterprise

Reviewers

design-authority, security-privacy, responsible-ai, evaluation-quality

Status

draft

Copilot Cowork

Microsoft Foundry plugin for Copilot Cowork, or this skill on its own: Download SKILL.md

Licence

CC BY 4.0, by Simon Foote

Purpose

A Foundry solution that works in a demonstration is not yet a service the organisation can rely on. This skill brings together, in one place, the evidence that the solution is good enough, safe enough, secure, affordable, observable, supportable and reversible, and turns it into a release recommendation the accountable owner can sign. It does not redo the specialist work; it checks that each piece exists, is current for the version being released, and has been accepted by the right people. Readiness goes wrong when it becomes a meeting rather than evidence, when the evaluation results relate to an earlier prompt or model version, and when nobody has rehearsed turning the solution off.

When to use

• A Foundry build is nearing its first production release.
• A pilot is about to widen its audience, add a surface or gain a write action.
• A partner is handing over a solution and the organisation must decide whether to accept it into service.
• A significant change, such as a new model, prompt or tool, needs a proportionate readiness check before release.

When not to use

• Deciding whether the solution should be built at all - use ai-foundry:foundry-solution-assessment.
• Assembling the formal evaluation evidence pack as a reviewer gate - that is the reviewer skill evaluation-quality:ai-release-evidence.
• Release readiness for a Microsoft 365 agent - use m365-agents:agent-release-readiness.
• Deployment readiness for an application built with Copilot Code - use copilot-code:code-deployment-readiness.
• Setting the organisation's release policy by risk tier - use platform-governance:ai-release-policy.
• Judging the impact of a change to a live solution - that is the reviewer skill design-authority:ai-change-impact-review.

Questions to ask first

1. Exactly which version is being released: model and version, prompt and instruction versions, tools, grounding sources and configuration?
2. What is the confirmed risk tier, and which reviewer gates does the release policy require for it?
3. Who is the accountable owner who will sign the release, and who supports the solution from the first day?
4. What happens if the solution must be switched off or rolled back in the first week, and has that been tried?
5. Which conditions, if any, would the owner accept as open at release, and who closes them by when?

Required inputs

• The solution design on the foundry-solution-design template, with every section completed for this version.
• Evaluation and safety results from ai-foundry:model-evaluation and ai-foundry:foundry-safety-evaluation, run on the release version.
• Reviewer findings for the gates the risk tier requires.
• The support and monitoring arrangements on the support-and-monitoring-plan template.

Where an item of evidence is missing or relates to a different version, mark it [To confirm: ...] and treat it as not met. Never accept a verbal assurance in place of evidence for a blocking item.

Decision framework

Each area is checked against evidence for the release version. Any unmet blocking item means no-go; advisory items may become conditions with an owner and date.

Practitioner approach

Pin the release version first. List the model and version, prompt and instruction versions, tool contracts, grounding sources, configuration and infrastructure versions that make up the release. Every piece of evidence is then checked against this list. The most common readiness failure is evaluation evidence from three prompt revisions ago presented as current.

Collect evidence, do not convene opinions. For each area in the framework, ask for the artefact: the evaluation report with the version on it, the security test results, the load test, the rollback rehearsal record. Where the evidence belongs to a reviewer gate, take the reviewer's recorded finding rather than re-reviewing. Your job is completeness and currency, not a second opinion on specialist work.

Rehearse the bad days. Before release, switch the solution off and back on in production or a production-equivalent environment, roll back one configuration change, and trigger one alert to confirm it reaches a person who acts. Run a deliberately failing scenario and have support explain it from the traces. These rehearsals find more real problems than any checklist, especially in solutions handed over by a partner.

Check ownership in practice. Ask the internal owner to make a small, safe change, such as updating an instruction or rotating a credential through the pipeline, and to find the runbook. If they cannot, the solution is not ready to be owned, whatever the contract says. Confirm the support route, the incident route through ai-operations:ai-incident-management, and the review calendar entry for model lifecycle and periodic evaluation.

Plan the release itself. Prefer a staged release to a defined audience with an agreed observation period and explicit criteria for widening, rather than switching everyone on at once. Confirm users will be told what the solution does, that it is AI and how to reach a person.

Then write the recommendation: go, go with conditions, or no-go. Conditions are advisory items with a named owner, a date and a check; blocking items are never converted into conditions to meet a deadline. Disagreement goes to the accountable executive through responsible-ai:ai-risk-acceptance, recorded in writing. State plainly what readiness cannot determine: behaviour on real traffic, which the observation period exists to reveal.

Outputs

• A production readiness record in the foundry-solution-design template: release version, evidence per area with status and source, blocking and advisory findings, conditions with owners and dates, rehearsal results and the recommendation.
• A staged release plan with audience, observation period and widening criteria.
• A signed release decision by the accountable owner.

Microsoft platform considerations

Foundry solutions depend on Azure resources, model deployments and service features whose availability, quota, lifecycle and preview status change. Confirm in the current source records that no production dependency is on a retired or soon-to-retire model or on an unsupported preview feature, or record the dependency as an accepted risk. Licensing questions route to the Microsoft licensing guide or a licensing partner.

Adoption considerations

Production release is when people start relying on the solution. Confirm the business owner has prepared users with what the solution does and does not do, how to give feedback and how to reach a person, and that the people whose work changes have been briefed.

Knowledge considerations

Confirm every grounding source in the release has an owner, a refresh route and a review date, and that the content precondition from the grounding strategy has been met. A solution released on unowned knowledge decays from launch.

Security and privacy considerations

Confirm security testing, identity review and privacy assessment are complete for the release version, that secrets are in managed stores, and that retention for threads, traces and indexes is set and enforced. The security-privacy gate findings are taken as recorded.

Responsible AI considerations

Confirm the impact assessment, human oversight design and transparency measures required for the risk tier are in place and tested. Residual risk that the owner accepts is recorded through responsible-ai:ai-risk-acceptance, never assumed.

Evaluation requirements

Quality and safety evaluation must have been run on the exact release version against thresholds agreed before results were seen, with the evidence pack accepted by evaluation-quality from tier 2. Any change after evaluation invalidates it until rerun.

Value measures

Confirm the baseline exists and the value measures can be collected from the first day, owned by the business owner. Track readiness findings that later caused incidents, to improve the checklist.

Scale notes

Starter

Use the framework as a one-page checklist with evidence links, rehearse the switch-off and rollback, and release to a small audience first. A partner may compile evidence; the internal owner signs.

Scale

Standardise the readiness record and rehearsals across Foundry solutions, automate evidence collection from pipelines and evaluation runs, and keep a log of conditions and their closure.

Enterprise

Integrate readiness with the organisation's release management and service transition processes, with reviewer gates scheduled into delivery plans and readiness evidence retained for audit against the AI management system.

Hand-offs

• To evaluation-quality:ai-release-evidence for the formal evidence pack from tier 2.
• To ai-operations:ai-support-model for transition into support.
• To ai-operations:ai-review-calendar for lifecycle and periodic evaluation reviews.
• To platform-governance:ai-solution-inventory to register the live solution.
• To responsible-ai:ai-risk-acceptance for residual risk the owner accepts.

Reviewer gates

The release policy sets the gates by risk tier. From tier 2, evaluation-quality accepts the evidence pack, security-privacy confirms security and privacy findings are closed, responsible-ai confirms oversight and transparency, and design-authority checks supportability. Each gate classifies findings as blocking or advisory, and a blocking finding stops release.

Failure modes

• Evaluation results from an earlier prompt or model version are accepted as evidence for the release.
• The kill switch and rollback were never rehearsed, and the first incident becomes an outage.
• A partner-built solution goes live without a handover, and nobody internal can operate it.
• Blocking findings are relabelled as conditions to hit a launch date, and the risk is never closed.
• The solution is switched on for everyone at once, and a problem reaches every user before it is seen.

Sources

references/sources.md in this skill lists the source records behind this skill, what each is authoritative for, how volatile it is and what you may do with it. Product availability, licensing, limits and preview status change often, so check the current source before stating them, and cite what you used.

Never

• Accept evidence that does not relate to the exact release version.
• Convert a blocking finding into a condition to meet a deadline.
• Release a solution whose switch-off and rollback have not been rehearsed.
• Recommend go without a named internal owner able to change and operate the solution.