Catalogue › AI Operations

CopilotForge - AI Operations

Use it in Copilot Cowork: download the AI Operations plugin for Copilot Cowork and upload it under Customize > Plugins > Upload plugin. How to install.

Operate deployed agents and AI solutions throughout support, monitoring, review, improvement and retirement.

Layer: assurance
Lead role: AI Operations Lead
Plugin name: copilotforge-ai-operations

Skill Status Lifecycle Description

ai-continuity draft Deliver, Launch, Improve Plans business continuity for work that depends on AI - impact of unavailability or forced switch-off, fallback to manual or simpler routes, decision rights for suspending a solution, user communications, preserving the skills and procedures people need without AI, and exit routes when a model or feature is withdrawn. Use when a solution becomes business-critical, before tier 3 release, or when a model or feature deprecation is announced.

ai-feedback-loop draft Measure, Improve Runs the feedback loop for live agents and AI solutions - collecting in-product ratings, comments, support tickets and Champion reports, triaging them by cause to knowledge, instruction, scope or training owners, acting on themes, and telling users what changed. Use when a solution goes live, when feedback piles up unread, or when users say nothing happens after they report a problem.

ai-incident-management draft Launch, Measure, Improve Runs incident management for AI failures and harms in live solutions - recognising AI incident types, setting severity, containing fast, preserving evidence, communicating, involving security, privacy and legal roles, and post-incident review before return to service. Use when an agent gives harmful, exposing or widely wrong outputs, takes a wrong action, or when AI incident procedures are being prepared.

ai-review-calendar draft Measure, Scale, Improve Maintains a calendar of scheduled reviews for every live agent and AI solution, with frequency set by risk tier, a standard agenda covering ownership, quality, usage, feedback, incidents, knowledge, risk tier and cost, and a recorded outcome to continue, improve, re-tier, escalate or retire. Use when live solutions have no review dates, when reviews keep slipping, or when nobody can say which solutions are still needed.

ai-solution-monitoring draft Launch, Measure, Improve Sets up operational monitoring for each live agent or AI solution - availability and errors, failed actions, usage by team, sampled answer quality, safety triggers, feedback, knowledge freshness and consumption - with thresholds, alert routing to named roles and a review rhythm. Use when a solution goes live, when users find failures before the owners do, or when alerts are so noisy that everyone ignores them.

ai-solution-retirement draft Improve Retires a live agent or AI solution safely once the decision is made - notifying users and pointing them to alternatives, removing sharing, disabling the solution, revoking connections, service identities and secrets, handling transcripts and logs under retention rules, cleaning up dependent flows and indexes, and closing the inventory record. Use when a review or owner decides to retire a solution, or when ownerless solutions accumulate.

ai-support-model draft Launch, Measure, Improve Designs the support model for live agents and AI solutions - support tiers from service desk to solution owner, knowledge owners, makers and platform team, AI-specific triage categories, response expectations by risk tier, runbooks and the handover from build to run - so every live solution has someone who answers and someone who fixes. Use when a solution is about to go live, or when issues bounce between teams with no owner.

knowledge-refresh draft Measure, Improve Keeps a live agent's knowledge current in operation - detecting source changes, moves and expiry, applying routine and urgent refreshes, confirming the solution picks up the change, retiring superseded content and re-running affected evaluation items. Use when a policy or document an agent relies on changes, when answers lag behind updated content, or when a knowledge library is moved or restructured under a live agent.

Skill	Status	Lifecycle	Description
`ai-continuity`	draft	Deliver, Launch, Improve	Plans business continuity for work that depends on AI - impact of unavailability or forced switch-off, fallback to manual or simpler routes, decision rights for suspending a solution, user communications, preserving the skills and procedures people need without AI, and exit routes when a model or feature is withdrawn. Use when a solution becomes business-critical, before tier 3 release, or when a model or feature deprecation is announced.
`ai-feedback-loop`	draft	Measure, Improve	Runs the feedback loop for live agents and AI solutions - collecting in-product ratings, comments, support tickets and Champion reports, triaging them by cause to knowledge, instruction, scope or training owners, acting on themes, and telling users what changed. Use when a solution goes live, when feedback piles up unread, or when users say nothing happens after they report a problem.
`ai-incident-management`	draft	Launch, Measure, Improve	Runs incident management for AI failures and harms in live solutions - recognising AI incident types, setting severity, containing fast, preserving evidence, communicating, involving security, privacy and legal roles, and post-incident review before return to service. Use when an agent gives harmful, exposing or widely wrong outputs, takes a wrong action, or when AI incident procedures are being prepared.
`ai-review-calendar`	draft	Measure, Scale, Improve	Maintains a calendar of scheduled reviews for every live agent and AI solution, with frequency set by risk tier, a standard agenda covering ownership, quality, usage, feedback, incidents, knowledge, risk tier and cost, and a recorded outcome to continue, improve, re-tier, escalate or retire. Use when live solutions have no review dates, when reviews keep slipping, or when nobody can say which solutions are still needed.
`ai-solution-monitoring`	draft	Launch, Measure, Improve	Sets up operational monitoring for each live agent or AI solution - availability and errors, failed actions, usage by team, sampled answer quality, safety triggers, feedback, knowledge freshness and consumption - with thresholds, alert routing to named roles and a review rhythm. Use when a solution goes live, when users find failures before the owners do, or when alerts are so noisy that everyone ignores them.
`ai-solution-retirement`	draft	Improve	Retires a live agent or AI solution safely once the decision is made - notifying users and pointing them to alternatives, removing sharing, disabling the solution, revoking connections, service identities and secrets, handling transcripts and logs under retention rules, cleaning up dependent flows and indexes, and closing the inventory record. Use when a review or owner decides to retire a solution, or when ownerless solutions accumulate.
`ai-support-model`	draft	Launch, Measure, Improve	Designs the support model for live agents and AI solutions - support tiers from service desk to solution owner, knowledge owners, makers and platform team, AI-specific triage categories, response expectations by risk tier, runbooks and the handover from build to run - so every live solution has someone who answers and someone who fixes. Use when a solution is about to go live, or when issues bounce between teams with no owner.
`knowledge-refresh`	draft	Measure, Improve	Keeps a live agent's knowledge current in operation - detecting source changes, moves and expiry, applying routine and urgent refreshes, confirming the solution picks up the change, retiring superseded content and re-running affected evaluation items. Use when a policy or document an agent relies on changes, when answers lag behind updated content, or when a knowledge library is moved or restructured under a live agent.

Credit

Created by Simon Foote as part of CopilotForge. Content is licensed under CC BY 4.0; keep the credit in each skill's license and authors fields when you share or adapt it. See NOTICE.md in the repository.