Catalogue › Automation › desktop-automation

Desktop automation

Decides when desktop flows that drive a user interface are justified for legacy systems with no connector or API, and sets the conditions for using them with care - attended or unattended runs, machines, credentials, fragile selectors, AI on screen content, ownership and an exit plan. Use when a step can only be done through a screen, when desktop flows keep breaking after updates, or when unattended runs are proposed.

Department

Automation

Lifecycle

Deliver, Assure, Improve

Maturity

builder, scaler, ai-enterprise

Risk tiers

1, 2, 3, 4

Platforms

power-platform, entra

Editions

starter, scale, enterprise

Reviewers

security-privacy, design-authority

Status

draft

Copilot Cowork

Automation plugin for Copilot Cowork, or this skill on its own: Download SKILL.md

Licence

CC BY 4.0, by Simon Foote

Purpose

Some systems offer no connector or API, only a screen, and desktop flows that click and type on a person's behalf can bridge the gap. They are also the most fragile, least secure and most expensive kind of automation to keep running. This skill decides when a desktop flow is justified and sets the conditions for using one: the run mode, the machine, the credentials, how screens are read, who owns it and how it will be retired. Desktop automation goes wrong when it is chosen because it is quick to record, when unattended bots run on shared or personal credentials, and when nobody plans for the day the target system changes.

When to use

• An automation assessment marks a step as user-interface-only, with no supported connector or API.
• A team proposes recording a desktop flow for a legacy, mainframe, terminal or vendor application.
• Desktop flows break after application or operating system updates, and support effort is rising.
• Unattended runs on a server or virtual machine are proposed, and credentials and machines need deciding.
• AI is proposed to read screen content, scanned documents or images inside a desktop flow.

When not to use

• Deciding which steps need automation, AI or a person - use automation:automation-assessment.
• Cloud flow structure, triggers and AI step validation - use automation:flow-design-patterns.
• Governing connectors and connections for cloud flows - use automation:automation-connector-governance.
• Building an API or integration layer for the legacy system instead - use ai-architecture:ai-integration-patterns.
• Retry and exception routing in detail - use automation:exception-handling.

Questions to ask first

1. Is there really no API, export, file drop, database view or vendor connector, and is one planned?
2. Will the flow run attended, alongside a person at their desk, or unattended on a machine with nobody watching?
3. Whose credentials will it use to sign in to the target system, and are they allowed to be used by software?
4. How often does the target application change its screens, and will the automation team be told in advance?
5. What happens to the business process if the desktop flow stops for a day, or a week?

Required inputs

• The automation assessment and design, with the user-interface-only steps marked.
• Confirmation from the system owner that no supported integration route exists, and whether one is planned.
• The target application's release pattern and the contact for advance notice of changes.
• Available machines and the organisation's rules for service accounts and unattended sign-in.
• The risk tier and data sensitivity.

If the system owner cannot confirm there is no integration route, ask the question formally before building. If the terms of use of the target system are unclear about automated access, route the question to the system owner or legal adviser rather than assuming.

Decision framework

Every desktop flow has an exit plan: the integration route that would replace it and the trigger for switching.

Practitioner approach

Challenge the premise first. Ask the system owner and the vendor, in writing, whether an API, a scheduled export, a file interface, a reporting database or a connector exists. Teams often reach for desktop flows because they do not know who to ask. A modest integration request can remove years of fragile screen automation.

If a desktop flow is justified, decide the run mode. Attended flows run while a person is signed in and watching; they use that person's access, and the person can step in when a screen looks wrong. They suit moderate volumes and processes where judgement is close at hand. Unattended flows run on a machine with nobody present; they need a dedicated machine or machine group, a dedicated account with only the access the process needs, and credentials held in a managed store rather than typed into the flow.

Build for brittleness. Prefer stable selectors (identifiers and names over screen positions and image matching), check that each expected screen has appeared before acting, and stop safely if it has not. Keep each desktop flow short and called from a cloud flow that handles orchestration, logging and exceptions. Long desktop flows are hard to debug and hard to restart.

Use AI on screen content with the same discipline as any AI step. Where text recognition or a model reads a scanned form or a screen region, ask for structured output, validate it and route low-confidence results to a person, following automation:flow-design-patterns. Never let a misread value be typed straight into a system of record.

Arrange change notice. Agree with the application owner that the automation team is told before screen changes, and schedule a regression run after every update to the target application, the operating system or the browser. Record each desktop flow in the solution inventory with its target application and version.

Plan for outage and exit. Document the manual fallback and who staffs it. Record the integration route that would replace the flow and review it at least yearly with the system owner. A desktop flow that has outlived its exit plan should be on the retirement list.

Outputs

• A desktop automation decision record: integration routes checked, justification, run mode, machine, account and credential store, screen stability, fallback and exit plan.
• A desktop flow specification: steps, selectors approach, screen checks, AI steps with validation, calling cloud flow and exception routes.
• An entry in the solution inventory with target application, version, owner and next review date.

Microsoft platform considerations

Power Automate desktop flows support attended and unattended runs, machine and machine group registration, credential integration with managed vaults and AI-based reading of documents and screens. Capabilities, run modes and licensing differ and change often. Check current behaviour in references/sources.md, and route licensing questions about unattended runs to the Microsoft licensing guide or a licensing partner.

Adoption considerations

Attended automations change the working day of the person beside them. Agree when the flow runs, what they should do if it stops and how to report problems. Do not present desktop flows as permanent; set expectations that they will be replaced when an integration route arrives.

Knowledge considerations

Not applicable to most desktop automation, because it drives screens rather than grounding on knowledge. Keep the screen-by-screen procedure and fallback instructions current, since they are the knowledge the manual fallback depends on.

Security and privacy considerations

Unattended accounts must be dedicated, least privilege and stored in a managed credential store; machines running unattended flows must be hardened, patched and monitored. Screen recordings and logs can capture personal data, so restrict access and set retention. Shared or personal credentials for unattended runs are not acceptable.

Responsible AI considerations

Where AI reads screens or scanned documents about people, misreads can produce wrong decisions about them. Validate and route low-confidence results to a person, and check that the model handles the range of documents and languages the process receives.

Evaluation requirements

Test against the current version of the target application with real screens, including error dialogs and slow responses. Re-run the regression set after every update to the application, operating system or browser, and before promotion.

Value measures

Track run success rate, failures by cause (screen change, timeout, credential), support effort and manual fallback use, at automation level. Compare the run cost with the cost of the integration route at each yearly review. The automation owner reports these to the process owner.

Scale notes

Starter

Use attended desktop flows only, for a small number of stable applications, called from cloud flows. Avoid unattended runs until accounts, machines and credential storage are properly set up.

Scale

Introduce unattended runs on managed machine groups with dedicated accounts and vaulted credentials. Keep a register of desktop flows by target application and review exit plans yearly.

Enterprise

Run desktop automation as a managed service with patching, capacity planning, change notice from application owners and a portfolio plan to replace desktop flows with integrations. Report fragility and cost across the estate.

Hand-offs

• To ai-architecture:ai-integration-patterns when an integration route should replace the desktop flow.
• To automation:exception-handling for screen failure and recovery routes.
• To automation:automation-monitoring for run health and alerting.
• To security-privacy:connector-review for unattended accounts and credential handling from tier 2.

Reviewer gates

From risk tier 2, security-privacy reviews unattended accounts, machines and credential storage, and shared or personal credentials in an unattended run are blocking. design-authority:platform-fit-review may challenge a desktop flow where an integration route exists.

Failure modes

• A desktop flow is recorded because it is quick, and an available API is never investigated.
• The vendor updates a screen, the flow types data into the wrong field and nobody notices for days.
• Unattended runs use a former employee's credentials or a shared account with broad access.
• No manual fallback is documented, and the process stops when the machine fails.

Sources

references/sources.md in this skill lists the source records behind this skill, what each is authoritative for, how volatile it is and what you may do with it. Product availability, licensing, limits and preview status change often, so check the current source before stating them, and cite what you used.

Never

• Build a desktop flow without first asking the system owner for an integration route.
• Run unattended flows on personal or shared credentials, or credentials typed into the flow.
• Type an unvalidated AI reading of a screen or document into a system of record.
• Leave a desktop flow without a manual fallback and an exit plan.