Designing Copilot Studio Agent ALM Across Dev, Test and Prod

1. Study log: the import works and the agent still breaks

This entry designs how a Copilot Studio agent moves from development to test to production. It covers the AB-100 objective “Design the ALM process for Copilot Studio agents, connectors, and actions.”[1]

I haven’t run a pipeline or imported an agent solution in a tenant, which is why the title says designing. Everything below comes from Microsoft Learn, plus a PowerShell script that checks a design file and two deployment settings files. All of them are synthetic. No environment, pipeline, agent or key vault was involved.

One list in Microsoft’s guidance shaped this design. Copilot Studio has items that aren’t solution-aware and “require post-deployment steps on the downstream environment”: Azure Application Insights settings, manual authentication settings, Direct Line / Web channel security settings, deployed channels and sharing.[3] The import page adds two more: you configure user authentication for the imported agent again, and you must publish the imported agent before it can be shared.[5] So a successful solution import can still leave you with an agent that doesn’t work, and the design treats those steps as part of every release, with a checklist per stage.

The second input: environment variables in Copilot Studio are read-only, and the published agent keeps the values it had when you published it. When an admin changes a value, you republish every agent that uses it. Secret variables are the exception because they’re read at runtime.[6]

Default approach	AI-hardened approach
Build the agent in the default solution.	Build in a custom solution with a custom publisher and prefix. Copilot Studio puts new agents in the default solution, and you need a custom solution to move them between environments.[3][4]
Hard-code the SharePoint site, queue name and API key.	Put settings in environment variables and connections in connection references.[3] Keep secrets in Azure Key Vault behind a secret environment variable.[16]
Export unmanaged and import it into production.	Deploy managed outside development. Pipelines don’t deploy unmanaged solutions.[3][7]
Let the maker deploy straight to production.	Use a delegated deployment with a service principal and an approval flow on the production stage.[8]
Treat a successful import as done.	Run the post-deployment checklist: authentication, Web channel security, channels, Application Insights, sharing and publish.[3][5]
Change an environment variable and walk away.	Republish every agent that reads a non-secret environment variable after its value changes.[6]
Plan rollback when you need it.	Turn on redeployment of past versions before the first release, and know that a redeploy is irreversible.[7][11]

Blueprint note. As of October 27, 2026, Microsoft’s study guide publishes only the “Skills measured as of November 16, 2026” list, with Plan at 35–40%, Design at 30–35% and Deploy at 20–25%.[1] The exam page says the English exam updates on November 16 and localized versions about eight weeks later.[1][2] This objective sits under “Design the ALM process for AI-powered business solutions,” which the change log marks as a minor change, inside the Deploy group whose share of the exam decreased.[1] If you test before November 16, the October 14 weights in the AB-100 blueprint entry apply. I use the November 16 objective name because that’s the only list Microsoft shows now, and I don’t mix the two sets of weights.

2. Architecture and prerequisites

The design

Four environments. Development and test are sandbox type, production is production type, as the Copilot Studio ALM guidance recommends.[3] A fourth production type environment is the pipelines host.[7] Each environment has its own Entra security group.[3]
One custom solution: the IT triage agent from the AB100-02 entry, its topics, three environment variables and two connection references, under a custom publisher with the prefix zk.
One pipeline with two stages, Test then Production. Both targets are managed environments.[7]
A delegated deployment on Production, where a service principal deploys after an approval flow says yes.[8]
One key vault per environment. The secret environment variable reads from the vault for its own environment.[16]
A settings file per stage. settings/test.json and settings/prod.json hold the values each stage needs, in the deployment settings format that pac solution create-settings generates.[13][14]
A post-deployment checklist per stage, generated by the script in section 4.

What a solution carries and what it doesn’t

Item	What Microsoft documents
Agent instructions	Included in the export.[5]
Custom topics and knowledge sources	Might not be included if they’re stored as separate resources or Dataverse tables.[5]
Bot properties	Conversation ID, CDS Bot ID, Environment ID and topic or node comments don’t transfer. The icon and channel details can be empty after import.[5]
Topics with a period in the name	The solution can’t be exported.[5]
Custom connectors	Import them first, then the connection reference with the agent solution.[5] They need a separate solution.[17]
Dataverse table data	Pipelines and solutions don’t carry data stored in Dataverse tables.[7]
Not solution-aware	Application Insights, manual authentication, Direct Line / Web channel security, deployed channels and sharing.[3]

Roles and licensing I verified

Who	What Microsoft documents
Maker exporting or importing	At least the System Customizer security role.[5] The solution explorer in Copilot Studio inherits the user’s security role privileges.[4]
Pipelines host	Should be a production environment. It doesn’t have to be a managed environment.[7]
Pipeline targets	Must be managed environments. Developer environments are the exception Microsoft names.[7] Managed environments need licenses that grant premium use rights.[7]
Delegated service principal	An S2S (application) user in the host and every target. Deployment Pipeline Administrator in the host, System Administrator in the targets. The person who configures it must own the enterprise application in Entra ID.[8]
Key Vault	The subscription needs the `PowerPlatform` resource provider registered. Users and the Dataverse service principal need Key Vault Secrets User, and Copilot Studio needs it for Microsoft Copilot Studio Service.[16]

From October 2026, the admin deployment page flags pipelines that deploy to unmanaged target environments. Admins get 30 days to enable managed environments, plus one 30-day extension per environment, and after that deployments to that target are blocked.[7][12] If your test environment isn’t managed today, fix that before you build the pipeline around it.

I didn’t verify Copilot Studio license requirements for makers beyond the roles above. Check your plans before you build.

3. Build steps and where each one fails

I haven’t run these steps in a tenant. They’re the order I’d build in, with the failure point Microsoft’s docs warn about.

Solution and components

Create a custom solution and set it as preferred. In the Copilot Studio solution explorer, select New solution, then Set preferred solution so new agents land there.[4] Use a custom publisher and prefix, and create separate solutions only when components must deploy independently.[3] Failure point: an agent that still lives in the default solution, which pipelines can’t run from.[7]
Add the agent and its dependencies. Select Add existing > Agent > Agent, then on the agent select Advanced > Add required objects.[5] Do the same for flows and environment variables.[5] Failure point: Microsoft says the most common reason an import fails is a missing required component.[5]
Check topic names. No periods in any topic name.[5] Failure point: a topic called “Reset v2.1 password” blocks the export.
Ship custom connectors on their own. Put each custom connector in its own solution and import it before the agent solution.[5][17] Failure point: a copy-environment operation breaks connection references for custom connectors, and you have to create new ones.[17]

Environment variables and connection references

Create environment variables without values in the solution. Microsoft’s answer to “Should I include the value in my solution?” is no. Include the definition and supply the value at deployment.[15] A current value wins over a default value.[15] Values and defaults are limited to 2,000 characters.[15] Failure point: a dev value that ships inside the solution and quietly becomes production’s value.
Map Copilot Studio types correctly. A JSON variable whose value isn’t JSON is a validation error. You see environment variable errors in the test chat and when publishing, not in the topic list.[6]
Bind connections through connection references. Flows in a solution use connection references for all connectors.[17] Name each one so you can tell them apart by name alone.[17] Failure point: whoever turns on a flow must own or have permission to use every connection in it, and OAuth connections can only be explicitly shared with a user that represents a service principal.[17]
Generate the settings files. Run pac solution create-settings --solution-zip <solution_zip_file_path> --settings-file <settings_file_name>, then fill in each stage’s values. Connection IDs come from connections you create in the target environment.[13] Failure point: during import, connection references are validated so the connections are usable by the connection reference owner. The connections must be owned by that owner or shared with them.[13]

Secrets

Create one key vault per environment. Microsoft suggests a separate vault for every Power Platform environment to limit the damage of a breach, and RBAC as the permission model.[16] The vault must be in the same tenant.[16] If the Key Vault firewall is on, allow the Power Platform IP addresses, because Power Platform isn’t included in “Trusted Services Only”.[16]
Let Copilot Studio read the secret. Grant Key Vault Secrets User to Microsoft Copilot Studio Service, then tag the secret version. AllowedBots takes values formatted as {envId}/{schemaName}, comma-separated for several agents. AllowedEnvironments takes an environment ID and allows all agents in it.[16] Failure point: the test pane shows “Bot isn’t allowed to use environment variable” until the tag exists.[16] Microsoft’s walk-through triggers that error with a Send a message node that shows the secret, and after tagging, the secret value appears in the test panel.[16] Delete that test node before you publish. I prefer AllowedBots for production because it names the one agent that needs the secret.

Pipeline

Set up the host and the stages. Use a production environment as the host. Using the same environment for development and the host isn’t supported.[7] Add Test, then Production. Failure point: a target environment that isn’t managed, which the October 2026 enforcement eventually blocks.[12]
Turn on redeployment of older versions. Without it, only higher versions can deploy, and rollback means downloading the artifact, raising the version in solution.xml and importing by hand.[7] Redeploy needs the Power Platform Pipelines package version 9.1.24023.240222117 or later in the pipelines host.[11] Failure point: Microsoft warns that redeploying “overwrites the most recent version and any data within that version” and calls it irreversible.[11] Back up the target before you roll back.
Delegate production. On the Production stage, check Is delegated deployment, select Service Principal and enter the client ID.[8] Build a cloud flow in the host on the OnApprovalStarted trigger that calls UpdateApprovalStatus with the service principal’s connection (20 approves, 30 rejects).[8] Failure point: no working approval flow, so every delegated deployment stays pending.[8] Don’t pick Stage Owner here. A stage owner can’t deploy solutions with connection references for OAuth connections.[8]
Gate the first stage. Pre-export Step Required runs your validation before the pipeline exports the solution, and Microsoft says to enable it only on the first stage.[9] I’d call the checker in section 4 from there.
Grant access on the first deployment. With Allow sharing requests, the requester names security groups for deployed objects. Sharing only happens the first time an object deploys, and Copilot sharing “might also be available” depending on region.[8] Failure point: picking the wrong group on day one. After that you manage access through the group.[8]

Deploy and finish

Deploy from the development environment. Pipelines validate the solution against the target first and ask for connection references and environment variables.[10] The same exported artifact moves through the stages in order, so nothing skips test.[7][10] Pipelines don’t publish unmanaged customizations before export, so publish objects as you save them.[7] Pipelines import with Upgrade and without Overwrite customizations.[7]
Run the post-deployment checklist. Reconfigure authentication, Web channel security, channels, Application Insights and sharing, then publish the agent.[3][5] The icon can take up to 24 hours to show everywhere.[5]
Republish after value changes. Any change to a non-secret environment variable needs a republish of the agents that read it.[6] On a production environment with Block unmanaged customizations on, you can still change environment variable values.[18]

Which automation tool

Microsoft’s Copilot Studio guidance compares three options.[3] Pipelines in Power Platform need little setup and suit citizen developers. GitHub Actions for Power Platform and Azure DevOps with Power Platform Build Tools take more setup and give you source control and CI/CD. My pick for an IT team with one agent solution is pipelines, with native Git integration for source control,[19] and a settings file per stage kept in the repo. I’d move to Azure DevOps or GitHub Actions if you need to deploy across tenants, which pipelines can’t do,[7] or to deploy several solutions in one run, which pipelines also can’t do.[7]

One thing to know about the settings files: pac solution import takes them with --settings-file,[14] but pac pipeline deploy has no settings-file parameter.[21] In the pipeline route, the files are the reviewed record of what each stage should get, and the deployer enters those values when prompted.

4. Engineering deliverable: ALM design review

The synthetic solution manifest

solution-manifest.json stands in for what you’d read out of the exported solution. The solution, agent, variables and connection references are made up. The connector IDs are the SharePoint and Azure DevOps IDs from Microsoft’s deployment settings example.[13]

{
  "schemaVersion": 1,
  "solution": {
    "uniqueName": "ZkItTriageAgent",
    "version": "1.0.4.0",
    "publisher": { "uniqueName": "zakitprolab", "prefix": "zk", "isCustom": true }
  },
  "agents": [
    {
      "schemaName": "zk_itTriageAgent",
      "topics": ["Greeting", "Check device compliance", "Escalate to service desk"],
      "environmentVariables": ["zk_KnowledgeSiteUrl", "zk_TriageConfig", "zk_DeskApiKey"]
    }
  ],
  "environmentVariables": [
    { "schemaName": "zk_KnowledgeSiteUrl", "type": "Text", "valueInSolution": false },
    { "schemaName": "zk_TriageConfig", "type": "JSON", "valueInSolution": false },
    { "schemaName": "zk_DeskApiKey", "type": "Secret", "valueInSolution": false }
  ],
  "connectionReferences": [
    { "logicalName": "zk_sharepoint_triage", "connectorId": "/providers/Microsoft.PowerApps/apis/shared_sharepointonline", "authType": "oauth" },
    { "logicalName": "zk_azuredevops_triage", "connectorId": "/providers/Microsoft.PowerApps/apis/shared_visualstudioteamservices", "authType": "oauth" }
  ],
  "customConnectors": []
}

The design file

alm-design.json records the environments, stages, delegated deployment, Key Vault plan and post-deployment steps. The devMarkers list is my own idea. It holds strings that should never appear in a test or production value.

{
  "schemaVersion": 1,
  "solutionUniqueName": "ZkItTriageAgent",
  "pipeline": {
    "name": "IT triage agent",
    "host": { "environment": "zk-pipelines-host", "type": "Production" },
    "allowRedeployOlderVersions": true
  },
  "environments": [
    { "name": "zk-dev", "role": "development", "type": "Sandbox", "managed": false, "securityGroup": "sg-zk-agent-makers", "keyVault": "kv-zk-dev" },
    { "name": "zk-test", "role": "target", "type": "Sandbox", "managed": true, "securityGroup": "sg-zk-agent-testers", "keyVault": "kv-zk-test" },
    { "name": "zk-prod", "role": "target", "type": "Production", "managed": true, "blockUnmanagedCustomizations": true, "securityGroup": "sg-zk-agent-users", "keyVault": "kv-zk-prod" }
  ],
  "stages": [
    { "name": "Test", "target": "zk-test", "previousStage": null, "deployAs": "managed", "settingsFile": "settings/test.json", "preExportStep": true, "delegated": null },
    { "name": "Production", "target": "zk-prod", "previousStage": "Test", "deployAs": "managed", "settingsFile": "settings/prod.json", "preExportStep": false,
      "delegated": { "type": "ServicePrincipal", "clientId": "11111111-2222-3333-4444-555555555555", "approvalFlow": true } }
  ],
  "secrets": [
    { "schemaName": "zk_DeskApiKey", "copilotStudioServiceRole": true,
      "perEnvironment": {
        "zk-test": { "vault": "kv-zk-test", "secretName": "desk-api-key", "allowTag": "AllowedBots" },
        "zk-prod": { "vault": "kv-zk-prod", "secretName": "desk-api-key", "allowTag": "AllowedBots" }
      } }
  ],
  "devMarkers": ["zk-dev", "ITTriageDev", "dev.contoso.example"],
  "postDeployment": {
    "applicationInsights": true,
    "manualAuthentication": true,
    "webChannelSecurity": true,
    "channels": true,
    "sharing": true,
    "publishAgent": true,
    "republishWhenEnvVarChanges": true
  }
}

The settings files

These use the deployment settings format.[13] The connection IDs are random hex. The secret values are placeholders that name the vault and secret. I didn’t find Microsoft’s documented value format for a secret variable in a settings file, so take the real format from your own pac solution create-settings output.

settings/test.json:

{
  "EnvironmentVariables": [
    { "SchemaName": "zk_KnowledgeSiteUrl", "Value": "https://contoso.sharepoint.com/sites/ITTriageTest" },
    { "SchemaName": "zk_TriageConfig", "Value": "{\"maxDevicesPerRequest\":5,\"escalationQueue\":\"Test-Queue\"}" },
    { "SchemaName": "zk_DeskApiKey", "Value": "kv-zk-test/desk-api-key" }
  ],
  "ConnectionReferences": [
    { "LogicalName": "zk_sharepoint_triage", "ConnectionId": "7f1c0a5e9b2d4c1a8e3f6b0d2a4c6e8f", "ConnectorId": "/providers/Microsoft.PowerApps/apis/shared_sharepointonline" },
    { "LogicalName": "zk_azuredevops_triage", "ConnectionId": "0b9d8c7e6f5a4b3c2d1e0f9a8b7c6d5e", "ConnectorId": "/providers/Microsoft.PowerApps/apis/shared_visualstudioteamservices" }
  ]
}

settings/prod.json:

{
  "EnvironmentVariables": [
    { "SchemaName": "zk_KnowledgeSiteUrl", "Value": "https://contoso.sharepoint.com/sites/ITTriage" },
    { "SchemaName": "zk_TriageConfig", "Value": "{\"maxDevicesPerRequest\":5,\"escalationQueue\":\"Service-Desk-L1\"}" },
    { "SchemaName": "zk_DeskApiKey", "Value": "kv-zk-prod/desk-api-key" }
  ],
  "ConnectionReferences": [
    { "LogicalName": "zk_sharepoint_triage", "ConnectionId": "3e5a7c9b1d2f4a6c8e0b2d4f6a8c0e2a", "ConnectorId": "/providers/Microsoft.PowerApps/apis/shared_sharepointonline" },
    { "LogicalName": "zk_azuredevops_triage", "ConnectionId": "9c8b7a6f5e4d3c2b1a0f9e8d7c6b5a4f", "ConnectorId": "/providers/Microsoft.PowerApps/apis/shared_visualstudioteamservices" }
  ]
}

The script

Test-AgentAlmDesign.ps1 reads the design, the manifest and each stage’s settings file. It writes a findings report and a post-deployment checklist per stage. It never connects to Power Platform, Copilot Studio, Dataverse, Azure or Key Vault.

Rule	Check	Source
E1	One development environment; host isn’t it; host is production type	[7]
E2	Production stage targets a production type environment; others aren’t production type	[3]
E3	Pipeline targets are managed environments	[7][12]
E4	Every environment has an Entra security group	[3]
E5	Production has Block unmanaged customizations on (Info)	[18]
P1	Stages run in order and don’t target the development environment	[7][10]
P2	Every stage deploys managed	[3][7]
P3	Production is delegated with an approval flow; no stage owner with OAuth connection references or a custom settings file	[8]
P4	Redeploying older versions is on	[7][11]
S1	Solution matches the design and uses a custom publisher	[3]
S2	No topic name contains a period	[5]
S3	No custom connector inside the agent solution	[5][17]
V1	Every environment variable has a value per stage	[13][15]
V2	No value stored in the solution	[15]
V3	Values are 2,000 characters or fewer	[15]
V4	JSON variables parse as JSON	[6]
V5	No dev marker in a test or production value	my rule
V6	Secrets use the stage’s own vault, an allow tag and the Copilot Studio Service role	[16]
C1	Every connection reference has a connection ID and the right connector ID	[13]
C2	No connection ID reused across stages	[13]
D1	Post-deployment steps listed, including republish after value changes	[3][5][6]

Safety features: -WhatIf writes nothing. A manifest stores SHA256 hashes of every input and output. A second run with the same inputs reports no changes. Changed inputs or an edited output stop the run unless you pass -Force. -Rollback removes only files whose hash still matches, and audit.log records each write and removal as JSON lines. The script exits with an error when the design has any Error finding, so a pre-export gate can call it.

#Requires -Version 7.2
<#
.SYNOPSIS
  Reviews an AB-100 ALM design for a Copilot Studio agent solution (environments, pipeline stages,
  per-stage deployment settings files, Key Vault secrets and post-deployment steps) and writes a
  findings report plus one post-deployment checklist per stage.
.NOTES
  Study aid. Reads local JSON files only. Never connects to Power Platform, Copilot Studio, Dataverse,
  Azure or Key Vault. All names, IDs and URLs in the sample files are synthetic.
#>
[CmdletBinding(SupportsShouldProcess)]
param(
    [Parameter(Mandatory)] [string] $DesignPath,
    [Parameter(Mandatory)] [string] $ManifestPath,
    [string] $OutputDirectory = './alm-review-output',
    [switch] $Force,
    [switch] $Rollback
)
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
$manifestFile = Join-Path $OutputDirectory 'manifest.json'
$auditPath = Join-Path $OutputDirectory 'audit.log'

function Get-Sha256([string] $Path) { (Get-FileHash -LiteralPath $Path -Algorithm SHA256).Hash.ToLowerInvariant() }
function Write-Audit([string] $Action, [string] $Detail) {
    if (-not (Test-Path -LiteralPath $OutputDirectory)) { return }
    $line = [ordered]@{ utc = [DateTime]::UtcNow.ToString('o'); user = [Environment]::UserName; action = $Action; detail = $Detail }
    Add-Content -LiteralPath $auditPath -Value ($line | ConvertTo-Json -Compress)
}
function Read-Json([string] $Path) {
    if (-not (Test-Path -LiteralPath $Path)) { throw "File not found: $Path" }
    try { Get-Content -LiteralPath $Path -Raw | ConvertFrom-Json -AsHashtable }
    catch { throw "Invalid JSON in ${Path}: $($_.Exception.Message)" }
}

# Rollback: remove only files this script wrote and nobody changed since.
if ($Rollback) {
    if (-not (Test-Path -LiteralPath $manifestFile)) { Write-Output "Nothing to roll back in $OutputDirectory."; return }
    $m = Get-Content -LiteralPath $manifestFile -Raw | ConvertFrom-Json
    $kept = 0
    foreach ($f in $m.files) {
        $p = Join-Path $OutputDirectory $f.name
        if (-not (Test-Path -LiteralPath $p)) { continue }
        if ((Get-Sha256 $p) -ne $f.sha256) { Write-Warning "Kept $($f.name): changed since this script wrote it."; $kept++; continue }
        if ($PSCmdlet.ShouldProcess($p, 'Remove generated file')) { Remove-Item -LiteralPath $p; Write-Audit 'remove' $f.name }
    }
    if ($kept -eq 0 -and $PSCmdlet.ShouldProcess($manifestFile, 'Remove manifest')) { Remove-Item -LiteralPath $manifestFile; Write-Audit 'remove' 'manifest.json' }
    return
}

$design = Read-Json $DesignPath
$solution = Read-Json $ManifestPath
if ($design['schemaVersion'] -ne 1 -or $solution['schemaVersion'] -ne 1) { throw 'Design and manifest must both use schemaVersion 1.' }
$baseDir = Split-Path -Parent (Resolve-Path -LiteralPath $DesignPath)

$findings = [System.Collections.Generic.List[object]]::new()
function Add-Finding([string] $Severity, [string] $Rule, [string] $Target, [string] $Message) {
    $findings.Add([pscustomobject][ordered]@{ Severity = $Severity; Rule = $Rule; Target = $Target; Message = $Message })
}

# ---- Environments and host ----
$envs = @{}; foreach ($e in @($design['environments'])) { $envs[$e['name']] = $e }
$devEnvs = @($design['environments'] | Where-Object { $_['role'] -eq 'development' })
if ($devEnvs.Count -ne 1) { Add-Finding 'Error' 'E1' 'environments' "Expected one development environment for the solution, found $($devEnvs.Count)." }
$hostEnv = $design['pipeline']['host']
if ($hostEnv['environment'] -in @($devEnvs | ForEach-Object { $_['name'] })) { Add-Finding 'Error' 'E1' $hostEnv['environment'] 'The pipelines host is also the development environment. That combination is not supported.' }
if ($hostEnv['type'] -ne 'Production') { Add-Finding 'Warning' 'E1' $hostEnv['environment'] 'The pipelines host should be a production type environment.' }
foreach ($e in $design['environments']) {
    if (-not $e['securityGroup']) { Add-Finding 'Warning' 'E4' $e['name'] 'No Entra security group limits access to this environment.' }
    if ($e['role'] -eq 'target' -and $e['managed'] -ne $true) { Add-Finding 'Error' 'E3' $e['name'] 'Pipeline target is not a managed environment. From October 2026 deployments to it are blocked after the grace period.' }
}

# ---- Stages ----
$stages = @($design['stages'])
$prodStage = $stages | Where-Object { $envs.ContainsKey($_['target']) -and $envs[$_['target']]['type'] -eq 'Production' } | Select-Object -Last 1
if (-not $prodStage) { Add-Finding 'Error' 'E2' 'stages' 'No stage targets a production type environment.' }
foreach ($e in $design['environments']) {
    if ($e['type'] -eq 'Production' -and $e['role'] -ne 'target') { continue }
    if ($e['type'] -eq 'Production' -and $prodStage -and $e['name'] -ne $prodStage['target']) { Add-Finding 'Warning' 'E2' $e['name'] 'Non-production environment uses the production type. Copilot Studio ALM guidance makes dev and test sandbox type.' }
}
if ($prodStage -and $envs[$prodStage['target']]['blockUnmanagedCustomizations'] -ne $true) { Add-Finding 'Info' 'E5' $prodStage['target'] 'Block unmanaged customizations is off, so makers can still make unmanaged changes in production.' }

$usesOAuth = @($solution['connectionReferences'] | Where-Object { $_['authType'] -eq 'oauth' }).Count -gt 0
for ($i = 0; $i -lt $stages.Count; $i++) {
    $s = $stages[$i]; $n = "stage:$($s['name'])"
    $expectedPrev = if ($i -eq 0) { $null } else { $stages[$i - 1]['name'] }
    if ($s['previousStage'] -ne $expectedPrev) { Add-Finding 'Error' 'P1' $n "previousStage is '$($s['previousStage'])', expected '$expectedPrev'. Stages must run in order." }
    if (-not $envs.ContainsKey($s['target'])) { Add-Finding 'Error' 'P1' $n "Target '$($s['target'])' isn't in the environments list."; continue }
    if ($envs[$s['target']]['role'] -eq 'development') { Add-Finding 'Error' 'P1' $n 'Target is a development environment in this host.' }
    if ($s['deployAs'] -ne 'managed') { Add-Finding 'Error' 'P2' $n 'Pipelines deploy managed solutions only. Deploy managed outside development.' }
    $d = $s['delegated']
    if ($s -eq $prodStage -and (-not $d -or $d['approvalFlow'] -ne $true)) { Add-Finding 'Warning' 'P3' $n 'Production stage has no delegated deployment with an approval flow.' }
    if ($d -and $d['type'] -eq 'StageOwner') {
        if ($usesOAuth) { Add-Finding 'Error' 'P3' $n 'Stage owner delegated deployments cannot deploy connection references for OAuth connections. Use a service principal.' }
        if ($s['settingsFile']) { Add-Finding 'Warning' 'P3' $n 'Stage owner delegated deployments do not take a custom DeploymentSettings.json in the maker experience.' }
    }
    if ($d -and $d['type'] -eq 'ServicePrincipal' -and $d['clientId'] -notmatch '^[0-9a-fA-F-]{36}$') { Add-Finding 'Error' 'P3' $n 'Service principal client ID is missing or not a GUID.' }
}
if ($design['pipeline']['allowRedeployOlderVersions'] -ne $true) { Add-Finding 'Warning' 'P4' 'pipeline' 'Redeploying older versions is off. Rollback then means editing solution.xml and importing by hand.' }
else { Add-Finding 'Info' 'P4' 'pipeline' 'Rollback = redeploy from run history. It overwrites the newer version and its data and cannot be undone.' }

# ---- Solution ----
$sol = $solution['solution']
if ($sol['uniqueName'] -ne $design['solutionUniqueName']) { Add-Finding 'Error' 'S1' $sol['uniqueName'] "Manifest solution doesn't match design solution '$($design['solutionUniqueName'])'." }
if ($sol['publisher']['isCustom'] -ne $true) { Add-Finding 'Error' 'S1' $sol['uniqueName'] 'Use a custom publisher and prefix.' }
foreach ($a in @($solution['agents'])) {
    foreach ($t in @($a['topics'] | Where-Object { $_ -like '*.*' })) { Add-Finding 'Error' 'S2' "$($a['schemaName'])/$t" 'Topic name contains a period. A solution with this agent cannot be exported.' }
}
foreach ($cc in @($solution['customConnectors'] | Where-Object { $_ })) {
    Add-Finding 'Error' 'S3' $cc 'Custom connector is in the agent solution. Ship it in its own solution and import it before the connection references.'
}

# ---- Per-stage settings files ----
$secretPlans = @{}; foreach ($sp in @($design['secrets'])) { if ($sp) { $secretPlans[$sp['schemaName']] = $sp } }
$allVaults = @($design['environments'] | ForEach-Object { $_['keyVault'] } | Where-Object { $_ })
$markers = @($design['devMarkers'] | Where-Object { $_ })
$connSeen = @{}
$agentVars = @($solution['agents'] | ForEach-Object { $_['environmentVariables'] } | Where-Object { $_ })
foreach ($v in @($solution['environmentVariables'])) {
    if ($v['valueInSolution'] -eq $true) { Add-Finding 'Warning' 'V2' $v['schemaName'] 'A value is stored in the solution. Remove it before export so each environment gets its own value.' }
}
foreach ($s in $stages) {
    if (-not $s['settingsFile']) { Add-Finding 'Error' 'V1' "stage:$($s['name'])" 'No settings file for this stage.'; continue }
    $sf = Join-Path $baseDir $s['settingsFile']
    $settings = Read-Json $sf
    $stageEnv = $envs[$s['target']]
    $evs = @{}; foreach ($x in @($settings['EnvironmentVariables'])) { if ($x) { $evs[$x['SchemaName']] = $x['Value'] } }
    $crs = @{}; foreach ($x in @($settings['ConnectionReferences'])) { if ($x) { $crs[$x['LogicalName']] = $x } }
    $known = @($solution['environmentVariables'] | ForEach-Object { $_['schemaName'] })
    foreach ($k in $evs.Keys | Where-Object { $_ -notin $known }) { Add-Finding 'Warning' 'V1' "$($s['name'])/$k" 'Settings file has a variable the solution does not define.' }

    foreach ($v in @($solution['environmentVariables'])) {
        $t = "$($s['name'])/$($v['schemaName'])"
        $val = $evs[$v['schemaName']]
        if ([string]::IsNullOrWhiteSpace($val)) { Add-Finding 'Error' 'V1' $t 'No value for this stage.'; continue }
        if ($val.Length -gt 2000) { Add-Finding 'Error' 'V3' $t "Value is $($val.Length) characters. The limit is 2,000." }
        if ($v['type'] -eq 'JSON') { try { $null = $val | ConvertFrom-Json -ErrorAction Stop } catch { Add-Finding 'Error' 'V4' $t 'JSON variable value does not parse as JSON.' } }
        foreach ($mk in $markers) { if ($val -like "*$mk*") { Add-Finding 'Error' 'V5' $t "Value contains the dev marker '$mk'." } }
        if ($v['type'] -eq 'Secret') {
            $plan = $secretPlans[$v['schemaName']]
            $p = if ($plan) { $plan['perEnvironment'][$s['target']] } else { $null }
            if (-not $p) { Add-Finding 'Error' 'V6' $t 'Secret has no Key Vault plan for this environment.'; continue }
            if ($p['vault'] -ne $stageEnv['keyVault']) { Add-Finding 'Error' 'V6' $t "Plan uses vault '$($p['vault'])', but this environment's vault is '$($stageEnv['keyVault'])'." }
            foreach ($other in $allVaults | Where-Object { $_ -ne $stageEnv['keyVault'] }) { if ($val -like "*$other*") { Add-Finding 'Error' 'V6' $t "Value points at another environment's vault '$other'." } }
            if ($v['schemaName'] -in $agentVars) {
                if ($p['allowTag'] -notin 'AllowedBots', 'AllowedEnvironments') { Add-Finding 'Error' 'V6' $t 'Agent reads this secret but the plan has no AllowedBots or AllowedEnvironments tag.' }
                if ($plan['copilotStudioServiceRole'] -ne $true) { Add-Finding 'Error' 'V6' $t 'Agent reads this secret but Microsoft Copilot Studio Service has no Key Vault Secrets User role in the plan.' }
            }
        }
    }

    $knownCr = @($solution['connectionReferences'] | ForEach-Object { $_['logicalName'] })
    foreach ($k in $crs.Keys | Where-Object { $_ -notin $knownCr }) { Add-Finding 'Warning' 'C1' "$($s['name'])/$k" 'Settings file has a connection reference the solution does not define.' }
    foreach ($c in @($solution['connectionReferences'])) {
        $t = "$($s['name'])/$($c['logicalName'])"
        $entry = $crs[$c['logicalName']]
        if (-not $entry -or [string]::IsNullOrWhiteSpace($entry['ConnectionId'])) { Add-Finding 'Error' 'C1' $t 'No connection ID for this stage.'; continue }
        if ($entry['ConnectorId'] -ne $c['connectorId']) { Add-Finding 'Error' 'C1' $t "ConnectorId '$($entry['ConnectorId'])' doesn't match the solution's '$($c['connectorId'])'." }
        $cid = $entry['ConnectionId']
        if ($connSeen.ContainsKey($cid) -and $connSeen[$cid] -ne $s['name']) { Add-Finding 'Error' 'C2' $t "Connection ID also used by stage '$($connSeen[$cid])'. Use a connection created in this stage's environment." }
        else { $connSeen[$cid] = $s['name'] }
    }
}
$vaultCounts = $allVaults | Group-Object | Where-Object Count -gt 1
foreach ($g in $vaultCounts) { Add-Finding 'Warning' 'V6' $g.Name 'Several environments share one key vault. Microsoft suggests a separate vault per environment.' }

# ---- Post-deployment steps (not solution-aware) ----
$pd = $design['postDeployment']
$labels = [ordered]@{
    applicationInsights = 'Azure Application Insights settings'
    manualAuthentication = 'Manual authentication settings'
    webChannelSecurity = 'Direct Line / Web channel security settings'
    channels = 'Deployed channels'
    sharing = 'Sharing with makers and end users'
    publishAgent = 'Publish the imported agent'
}
foreach ($k in $labels.Keys) { if ($pd[$k] -ne $true) { Add-Finding 'Error' 'D1' 'postDeployment' "Missing post-deployment step: $($labels[$k])." } }
$nonSecretAgentVars = @($solution['environmentVariables'] | Where-Object { $_['schemaName'] -in $agentVars -and $_['type'] -ne 'Secret' })
if ($nonSecretAgentVars.Count -and $pd['republishWhenEnvVarChanges'] -ne $true) { Add-Finding 'Error' 'D1' 'postDeployment' 'Agent reads non-secret environment variables, so it must be republished after their values change.' }

# ---- Report and checklists ----
$errorCount = @($findings | Where-Object Severity -eq 'Error').Count
$report = [ordered]@{
    design = Split-Path $DesignPath -Leaf
    solution = "$($sol['uniqueName']) $($sol['version'])"
    errors = $errorCount
    warnings = @($findings | Where-Object Severity -eq 'Warning').Count
    findings = @($findings)
}
$checklists = [ordered]@{}
foreach ($s in $stages) {
    $lines = @("# Post-deployment checklist: $($sol['uniqueName']) $($sol['version']) to $($s['target'])", '')
    foreach ($k in $labels.Keys) { $lines += "- [ ] $($labels[$k])" }
    foreach ($v in $nonSecretAgentVars) { $lines += "- [ ] Republish the agent after any change to $($v['schemaName'])" }
    foreach ($sp in $secretPlans.Values) { $p = $sp['perEnvironment'][$s['target']]; if ($p) { $lines += "- [ ] Key Vault $($p['vault']): tag $($p['allowTag']) on secret $($p['secretName'])" } }
    $checklists["postdeploy-$($s['name'].ToLowerInvariant()).md"] = ($lines -join "`n") + "`n"
}

# ---- Output with idempotency ----
$inputFiles = @($DesignPath, $ManifestPath) + @($stages | Where-Object { $_['settingsFile'] } | ForEach-Object { Join-Path $baseDir $_['settingsFile'] })
$inputHash = ($inputFiles | ForEach-Object { Get-Sha256 $_ }) -join ''
$outNames = @('alm-report.json') + @($checklists.Keys)
$skipWrite = $false
if (Test-Path -LiteralPath $manifestFile) {
    $old = Get-Content -LiteralPath $manifestFile -Raw | ConvertFrom-Json
    $intact = @($old.files | Where-Object { $p = Join-Path $OutputDirectory $_.name; (Test-Path -LiteralPath $p) -and (Get-Sha256 $p) -eq $_.sha256 }).Count -eq @($old.files).Count
    if ($old.inputSha256 -eq $inputHash -and $intact) { Write-Output 'No changes: output already matches these inputs.'; $skipWrite = $true }
    elseif (-not $Force) { throw "Output in $OutputDirectory came from different input or was edited. Re-run with -Force." }
}
if (-not $skipWrite -and $PSCmdlet.ShouldProcess($OutputDirectory, "Write report with $($findings.Count) findings and $($checklists.Count) checklists")) {
    if (-not (Test-Path -LiteralPath $OutputDirectory)) { New-Item -ItemType Directory -Path $OutputDirectory | Out-Null }
    $report | ConvertTo-Json -Depth 6 | Set-Content -LiteralPath (Join-Path $OutputDirectory 'alm-report.json') -Encoding utf8
    foreach ($k in $checklists.Keys) { Set-Content -LiteralPath (Join-Path $OutputDirectory $k) -Value $checklists[$k] -Encoding utf8 -NoNewline }
    $files = @($outNames | ForEach-Object { [ordered]@{ name = $_; sha256 = Get-Sha256 (Join-Path $OutputDirectory $_) } })
    [ordered]@{ inputSha256 = $inputHash; files = $files } | ConvertTo-Json -Depth 4 | Set-Content -LiteralPath $manifestFile -Encoding utf8
    Write-Audit 'write' "findings=$($findings.Count) errors=$errorCount files=$($outNames.Count)"
}
$findings | Sort-Object Severity, Rule, Target | Format-Table -AutoSize -Wrap | Out-String -Width 160 | Write-Output
if ($errorCount -gt 0) { throw "$errorCount design error(s). See $(Join-Path $OutputDirectory 'alm-report.json')." }

What I tested

All runs used the synthetic files above on Linux with PowerShell 7.4.6, except the row that names 7.6.7.

Test	Result
`-WhatIf` on the good design	Printed `Write report with 1 findings and 2 checklists` and created no output folder.
Normal run on the good design	Exit 0. One Info finding (P4, the redeploy warning). Wrote `alm-report.json`, two checklists, `manifest.json` and `audit.log`.
Second run, same inputs	`No changes: output already matches these inputs.`
Edited `alm-report.json`, run again	Stopped with “came from different input or was edited. Re-run with -Force.”
Same, with `-Force`	Rebuilt the output. Exit 0.
`-Rollback -WhatIf`	Listed four removals and removed nothing.
Edited a checklist, then `-Rollback`	Removed the untouched files and kept the edited one with a warning. The manifest stayed because a tracked file remained.
`-Force`, then a clean `-Rollback`	Only `audit.log` remained, with 9 JSON lines covering every write and removal.
Production settings copied from test	3 errors: both connection IDs reused across stages (C2) and the secret pointing at the test vault (V6).
Broken design, manifest and settings	17 errors, 7 warnings and 1 Info across 17 rules. P1, V1 and V3 weren’t triggered by this variant.
Invalid JSON in the design file	Stopped with `Invalid JSON in bad.json` and created no output folder.
PowerShell 7.6.7	Same results for the good and broken designs.
PSScriptAnalyzer 1.25.0	No errors or warnings. 41 Information findings, all `PSAvoidUsingPositionalParameters`.

Not tested: anything in a tenant. That includes pac solution create-settings output, a real import, pipeline prevalidation, approval flows, service principal deployments, Key Vault tags, republishing and rollback.

5. Public-objective concepts to retain

The objective name comes from the November 16, 2026 skills list:[1]

“Design the ALM process for Copilot Studio agents, connectors, and actions.” Agents travel in custom solutions,[4] at least three environments with production as production type,[3] managed outside development,[3][7] environment variables for settings and connection references for connections,[3] and pipelines, GitHub Actions or Azure DevOps to deploy.[3]
What doesn’t travel: Application Insights, manual authentication, Web channel security, channels and sharing need post-deployment steps,[3] and an imported agent needs authentication again and a publish.[5]
When values take effect: non-secret environment variables are baked in at publish, so you republish after a change. Secrets are read at runtime.[6]
Control over the deployment itself: delegated deployments with a service principal and an approval flow,[8] gated extensions,[9] the same artifact through every stage,[7] and managed target environments.[7][12]
Testing: Microsoft’s testing guidance says to run tests before production and consider automated testing in the pipeline, and to check that production knowledge sources are referenced if dev and test used different ones.[20] The same group lists ALM for Microsoft Foundry Agents Service and custom AI models as separate objectives.[1] This entry covers Copilot Studio only.

6. Next step in the study journey

Next is observability and FinOps for agents: telemetry, drift and failure recovery, for the objectives under “Analyze, monitor, and tune AI-powered business solutions.”[1] Application Insights is on this entry’s post-deployment list for a reason. For the tool review that belongs in the pre-export gate, see the MCP and computer-use entry. For environment and data policy basics, see the AB-900 agent lifecycle entry. The AB-100 certification guide covers whether the Expert credential fits your career.

Sources

Designing Copilot Studio Agent ALM Across Dev, Test and Prod

Top 10 Intune and Autopilot Enrollment Errors: Causes and Fixes