Azure Agent Skills Ecosystem: Three-Repo Analysis (githubcopilotforAzure vs microsoft/skills vs MicrosoftDocs/Agent-Skills)
Date: March 11, 2026 Scope:
azure-skills|microsoft/skills|MicrosoftDocs/Agent-SkillsPurpose: Strategic comparison, contribution flows, automation gaps, and convergence plan
| Dimension | azure-skills |
microsoft/skills |
MicrosoftDocs/Agent-Skills |
|---|---|---|---|
| Purpose | Azure operations skill pack for GitHub Copilot | Multi-agent SDK coding patterns library | Auto-generated docs-to-skills catalog |
| Skill count | ~24 skills | ~140 skills (6 languages) | ~193 skills |
| Content depth | Deep — inline runbooks, scripts, MCP tool orchestration | Medium — SDK patterns, acceptance criteria, code templates | Shallow — categorized link tables to Microsoft Learn |
| Content type | Operational workflows (deploy, diagnose, manage) | SDK coding patterns per language | Documentation index (no inline content) |
| Target platforms | VS Code, VS 2022/2026, GitHub Copilot CLI, Claude Code | GitHub Copilot, Claude Code, OpenCode | All agents (Claude Code, Copilot, Gemini CLI, Cursor, Codex, etc.) |
| Language coverage | Language-agnostic (infra/ops) | Python (41), .NET (28), TypeScript (25), Java (25), Go (8), Rust (7) | Language-agnostic (service-level) |
| Authoring model | Hand-crafted, PR-based | Hand-crafted, PR-based | Auto-generated pipeline (docs2skills/1.0.0) from Microsoft Learn |
| Spec compliance | agentskills.io | agentskills.io | agentskills.io |
| License | MIT | MIT | CC BY 4.0 + MIT |
| Update frequency | On-demand via PR; publishes weekdays 11am UTC | On-demand via PR | Weekly automated crawl |
┌─────────────────────────────────────────────────────────────────────┐
│ DEVELOPER JOURNEY │
│ │
│ "What Azure services "Write code using "Deploy, monitor, │
│ exist? How do they Azure SDKs" and manage my │
│ work?" production app" │
│ │
│ ┌───────────────────┐ ┌───────────────────┐ ┌──────────────┐ │
│ │ MicrosoftDocs/ │ │ microsoft/ │ │ azure-skills │ │
│ │ Agent-Skills │ │ skills │ │ │ │
│ │ │ │ │ │ │ │
│ │ LAYER: Discovery │ │ LAYER: Build │ │ LAYER: Run │ │
│ │ & Reference │ │ & Code │ │ & Operate │ │
│ │ │ │ │ │ │ │
│ │ 193 services │ │ 132 SDK skills │ │ 24 deep ops │ │
│ │ ALL platforms │ │ 6 languages │ │ skills │ │
│ │ Auto-generated │ │ Hand-crafted │ │ Hand-crafted │ │
│ └───────────────────┘ └───────────────────┘ └──────────────┘ │
│ │
│ BREADTH ◄─────────────────────────────────────────────► DEPTH │
└─────────────────────────────────────────────────────────────────────┘
Layer 1 — Discovery & Reference (MicrosoftDocs/Agent-Skills)
- Answers: "What is this Azure service? What are its limits? How is it architected?"
- Provides categorized links to Learn docs (Troubleshooting, Best Practices, Architecture, Limits & Quotas, Security, Configuration, Integrations, Deployment)
- Requires network access to fetch actual content at runtime
- Zero code, zero scripts — purely a documentation index
Layer 2 — Build & Code (microsoft/skills)
- Answers: "How do I write correct code against this Azure SDK?"
- Provides language-specific SDK patterns, install commands, authentication flows, anti-patterns
- Includes acceptance criteria defining correct vs. incorrect code (calibrated to LLM hallucination patterns)
- Targets developers actively writing application code
Layer 3 — Run & Operate (azure-skills)
- Answers: "How do I deploy, diagnose, optimize, and secure my Azure workloads?"
- Provides operational runbooks with plan-first workflows, MCP tool orchestration, error recovery
- Covers deployment (azd, Bicep, Terraform), diagnostics (KQL, App Insights), cost optimization, RBAC, compliance
- Targets teams running production workloads
| Prompt | Why this repo |
|---|---|
| "What are the limits of Azure Functions consumption plan?" | Service limits/quotas — categorized in the skill |
| "Best practices for Cosmos DB partitioning" | Architecture best practices — links to Learn docs |
| "How do I configure Azure Front Door with WAF?" | Configuration reference — categorized links |
| "What's the difference between Azure Service Bus and Event Grid?" | Decision-making category in both skills |
| "Troubleshoot Azure Bastion connectivity issues" | Troubleshooting category — no operational skill exists for Bastion |
| "Azure Kubernetes Service security hardening" | Security category in the AKS skill |
| Prompt | Why this repo |
|---|---|
| "Build a Python agent using Azure AI Foundry" | azure-ai-agents-py — post-cutoff SDK, LLM will hallucinate without it |
| "Add Azure Service Bus to my .NET app" | azure-service-bus-dotnet — correct client selection, auth patterns |
| "Integrate Cosmos DB with my FastAPI backend" | azure-cosmos-py — Pydantic model hierarchy, parameterized queries |
| "Use Azure Identity in my Java application" | azure-identity-java — DefaultAzureCredential patterns |
| "Set up blob storage in TypeScript with streaming" | azure-storage-blob-ts — streaming patterns, error handling |
| "Create a voice agent with Azure Communication Services" | azure-communication-voice-live-py — brand new SDK |
| Prompt | Why this repo |
|---|---|
| "Deploy my app to Azure Container Apps" | azure-deploy — plan-first workflow, MCP tool orchestration |
| "Why is my App Service returning 503 errors?" | azure-diagnostics — KQL queries, App Insights correlation |
| "Optimize my Azure spending" | azure-cost-optimization — cost analysis, rightsizing recommendations |
| "Set up RBAC for my team's subscription" | azure-rbac — role assignment workflows |
| "Create infrastructure for a microservices app" | azure-prepare — Bicep/Terraform/AZD scaffolding |
| "Query my Application Insights logs for slow requests" | azure-kusto — KQL query authoring |
| "Visualize my Azure resource topology" | azure-resource-visualizer — resource graph visualization |
| "Check my app's compliance with our governance policies" | azure-compliance — compliance checks |
| Prompt | Could be handled by | Best repo |
|---|---|---|
| "Set up Azure AI Search in Python" | All three | microsoft/skills (SDK code patterns) |
| "Debug Event Hub message loss" | Agent-Skills + azure-skills | azure-skills (diagnostics) + microsoft/skills (correct consumer pattern) |
| "What auth should I use for Azure Storage?" | All three | microsoft/skills (DefaultAzureCredential with language-specific code) |
| Process | Automated? | How |
|---|---|---|
| Skill generation | Yes | docs2skills/1.0.0 Azure DevOps pipeline crawls Microsoft Learn |
| Content updates | Yes | Weekly automated crawl regenerates skills |
| Quality validation | Partial | Pipeline classifies and structures; no automated eval |
| Publishing | Yes | Pipeline commits directly to GitHub |
| Contribution | N/A | External contributions explicitly not accepted |
| Process | Automated? | How |
|---|---|---|
| Skill creation | No | Manual authoring following skill-creator guide |
| Content updates | No | Manual PRs when SDK changes |
| Quality validation | Partial | CI runs typecheck, unit tests, mock-based skill evals (non-blocking) |
| SDK version tracking | No | Versions hardcoded in markdown |
| Publishing | No | Manual PR merge to main |
| Nightly evals | Yes | Copilot SDK tests at 2am UTC (non-blocking) |
| Docs site | Yes | GitHub Pages auto-deployed on merge |
| Process | Automated? | How |
|---|---|---|
| Skill creation | No | Manual authoring, copy from tests/_template/ |
| Content updates | No | Manual PRs with version bump |
| Quality validation | Yes | Strict CI: ESLint, TypeScript, token limits, frontmatter, version bump, reference integrity |
| Integration testing | Yes | Nightly scheduled runs against real Azure (3 time slots) |
| Publishing | Yes | Auto-sync to microsoft/azure-skills weekdays 11am UTC via rsync |
| SDK version tracking | No | Versions hardcoded in references/sdk/*.md |
| Token analysis | Yes | Auto-posted to PRs with per-file breakdowns |
MicrosoftDocs/ microsoft/ azure-skills
Agent-Skills skills
───────────── ────────── ─────────────
Skill Generation ██████████ AUTO ░░░░░░░░░░ MANUAL ░░░░░░░░░░ MANUAL
Content Updates ██████████ AUTO ░░░░░░░░░░ MANUAL ░░░░░░░░░░ MANUAL
CI Validation ████░░░░░░ PARTIAL ████░░░░░░ PARTIAL ██████████ STRICT
Integration Tests ░░░░░░░░░░ NONE ████████░░ NIGHTLY ██████████ NIGHTLY
Publishing ██████████ AUTO ░░░░░░░░░░ MANUAL ██████████ AUTO
SDK Drift Detection ░░░░░░░░░░ NONE ░░░░░░░░░░ NONE ░░░░░░░░░░ NONE
██ = Automated ░░ = Manual/Missing
╔══════════════════════════════════════════════════════════════════════╗
║ SDK DEVELOPER SHIPS AN UPDATE ║
║ (new version, new service, breaking change) ║
╚══════════════════════╤═══════════════════════════════════════════════╝
│
▼
┌──────────────────────────────┐
│ STEP 1: UPDATE LEARN DOCS │
│ learn.microsoft.com/azure/ │
│ │
│ • Quickstarts │
│ • Best practices │
│ • Troubleshooting │
│ • Limits & quotas │
│ • API reference │
└──────────────┬───────────────┘
│
┌─────────────┼─────────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌──────────────┐ ┌──────────────────┐
│ MicrosoftDocs│ │ microsoft/ │ │ azure-skills │
│ /Agent-Skills│ │ skills │ │ │
│ │ │ │ │ │
│ WAIT │ │ MANUAL PR │ │ MANUAL PR │
│ │ │ │ │ (if operational) │
└──────┬──────┘ └──────┬───────┘ └────────┬─────────┘
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌──────────────┐ ┌──────────────────┐
│ Weekly crawl │ │ For EACH │ │ Update if service │
│ auto-detects │ │ language: │ │ has operational │
│ new/updated │ │ │ │ skill: │
│ Learn pages │ │ 1. Update │ │ │
│ │ │ SKILL.md │ │ 1. Update SKILL.md│
│ OR │ │ 2. Update │ │ 2. Update refs/ │
│ │ │ refs/sdk/ │ │ sdk/*.md │
│ File issue │ │ 3. Update │ │ 3. Bump version │
│ for re-crawl │ │ acceptance│ │ 4. Update tests │
│ │ │ criteria │ │ 5. PR → CI gates │
│ │ │ 4. Update │ │ │
│ │ │ scenarios │ │ If NEW service │
│ │ │ 5. PR → CI │ │ with ops needs: │
│ │ │ │ │ │
│ No PR needed │ │ Repeat ×5 │ │ 1. Create skill │
│ │ │ languages │ │ directory │
│ │ │ potentially │ │ 2. Write SKILL.md│
│ │ │ │ │ 3. Add references│
└──────┬──────┘ └──────┬───────┘ │ 4. Create tests │
│ │ │ 5. Register in │
│ │ │ skills.json │
│ │ │ 6. PR → CI gates │
│ │ └────────┬─────────┘
▼ ▼ ▼
┌─────────────┐ ┌──────────────┐ ┌──────────────────┐
│ AUTO: │ │ MANUAL: │ │ AUTO: │
│ Pipeline │ │ Maintainer │ │ Publishes to │
│ regenerates │ │ reviews & │ │ microsoft/azure- │
│ SKILL.md │ │ merges │ │ skills weekdays │
│ with updated │ │ │ │ 11am UTC │
│ Learn URLs │ │ No auto- │ │ │
│ │ │ publish │ │ │
└──────────────┘ └──────────────┘ └──────────────────┘
EFFORT: ~0 hours ~2-8 hours/language ~4-16 hours
(automated) (manual per language) (if applicable)
| Repo | Action Required | Effort |
|---|---|---|
MicrosoftDocs/Agent-Skills |
None — Learn URLs don't change | 0 |
microsoft/skills |
Update version in references/sdk/*.md, update patterns if new APIs added, run pnpm harness <skill> --mock |
1-2 hours per language |
azure-skills |
Update references/sdk/*.md if version referenced, bump skill version |
1 hour (if applicable) |
| Repo | Action Required | Effort |
|---|---|---|
MicrosoftDocs/Agent-Skills |
Update Learn docs first; wait for re-crawl | 0 (after Learn update) |
microsoft/skills |
Update SKILL.md, acceptance criteria (add old pattern as INCORRECT), update all test scenarios, across all affected languages | 4-8 hours per language |
azure-skills |
Update if any operational skill references the SDK pattern | 2-4 hours (if applicable) |
| Repo | Action Required | Effort |
|---|---|---|
MicrosoftDocs/Agent-Skills |
Publish Learn docs under /azure/; pipeline auto-generates skill within a week |
0 (after Learn docs exist) |
microsoft/skills |
Create new skill per language: SKILL.md + acceptance criteria + test scenarios + symlinks + README update | 4-8 hours per language × N languages |
azure-skills |
Create operational skill only if service needs deploy/diagnose/manage workflows | 8-16 hours (if applicable) |
Yes, but the value is concentrated in specific artifacts.
| Artifact | Why it matters | Example |
|---|---|---|
| Acceptance criteria (correct/incorrect code pairs) | Calibrated to actual LLM hallucination modes — no other repo has this | PersistentAgentsClient not AgentsClient; parameterized queries not f-strings |
| Post-training-cutoff SDK patterns | LLMs literally don't know these APIs; they hallucinate from older/similar APIs | Azure AI Agents SDK, Voice Live, Foundry agents |
| Anti-pattern documentation | Prevents common mistakes that aren't in official docs | "Never use EventHubConsumerClient in production" |
| Test/eval infrastructure | Unique evaluator scoring generated code against acceptance criteria | tests/harness/evaluator.ts with 0-100 scoring |
| Artifact | Why it overlaps | Better source |
|---|---|---|
| Mature SDK quickstart content | Already in LLM pretraining (Storage, Event Hubs, Service Bus basics) | LLM pretraining + MicrosoftDocs/Agent-Skills |
| Cloud architecture patterns | Restated from Azure Architecture Center | MicrosoftDocs/Agent-Skills links directly to source |
| Basic auth patterns | DefaultAzureCredential is well-known to all LLMs |
LLM pretraining |
| Install commands | pip install azure-X is trivially derivable |
Package registries |
microsoft/skillsearns its place as an anti-hallucination layer for newer SDKs. Its unique value proposition is: "We know what LLMs get wrong with these SDKs, and we encode the correct patterns alongside the specific hallucinations to avoid." For mature, well-documented services, the marginal value overMicrosoftDocs/Agent-Skills+ LLM pretraining is thin.
Yes, in tiers. The infrastructure is approximately 80% there — the gaps are in monitoring, detection, and orchestration.
| Component | Exists in | Could support |
|---|---|---|
| Skill parser (YAML frontmatter extraction) | azure-skills (scripts/src/shared/parse-skill.ts) |
SDK inventory building |
| Reference validator (broken link detection) | azure-skills (scripts/src/references/cli.ts) |
Stale SDK URL detection |
| Token budget checker | azure-skills (scripts/src/tokens/) |
Post-update validation |
| Integration test runner | Both repos | Regression testing after updates |
| Publish pipeline (rsync + PR) | azure-skills |
Triggered updates |
| Mock-based eval harness | microsoft/skills (tests/harness/) |
Automated quality checks |
Azure SDK Release CSV
(github.com/Azure/azure-sdk/_data/releases/latest)
│
▼
┌─────────────────────────────┐
│ 1. VERSION MONITOR │ ← MISSING
│ Poll the CSV version │ Use Azure/azure-sdk release
│ mapping for new Azure │ CSVs (python-packages.csv,
│ SDK releases │ dotnet-packages.csv, js-packages.csv,
│ │ java-packages.csv, go-packages.csv)
└────────────┬────────────────┘
│
▼
┌─────────────────────────────┐
│ 2. SDK INVENTORY │ ← MISSING
│ Centralized map of │ Versions hardcoded in markdown
│ skill → SDK packages │ No structured dependency file
│ → pinned versions │
└────────────┬────────────────┘
│
▼
┌─────────────────────────────┐
│ 3. CHANGE CLASSIFIER │ ← BUILDABLE (data already exists)
│ │
│ Primary: Parse the monthly │ Monthly YAML files at
│ YAML ChangelogContent │ _data/releases/YYYY-MM/<lang>.yml
│ field for section headers: │ contain inline changelog content
│ "Breaking Changes", │ with standardized sections
│ "Features Added", │ (~70% of entries populated)
│ "Bugs Fixed" │
│ │ Fallback: fetch full CHANGELOG.md
│ Secondary: Compare semver │ via ChangelogUrl field for the
│ major bump (1.x → 2.x) │ ~30% with empty ChangelogContent
│ │
│ Tertiary: Check CSV │ CSV Replace/ReplaceGuide columns
│ Support="deprecated" and │ for package-level breaking changes
│ Replace fields │
└────────────┬────────────────┘
│
┌──────┴──────┐
▼ ▼
┌──────────────┐ ┌──────────────┐
│ Minor/Patch │ │ Breaking │
│ │ │ Change │
│ Auto-update │ │ Flag for │
│ version refs │ │ human review│
└──────┬───────┘ └──────┬───────┘
│ │
▼ ▼
┌─────────────────────────────┐
│ 4. CONTENT GENERATOR │ ← PARTIALLY EXISTS
│ LLM-assisted update of │ docs2skills pattern adaptable
│ SKILL.md + refs/sdk/ │ Token checker exists
│ + acceptance criteria │ Frontmatter validator exists
└────────────┬────────────────┘
│
▼
┌─────────────────────────────┐
│ 5. VALIDATE & PR │ ← EXISTS
│ Token checks │ CI pipelines in both repos
│ Eval harness │ Integration tests
│ Integration tests │ Auto-PR creation
│ Create PR │
└─────────────────────────────┘
Primary source: Monthly YAML ChangelogContent field
Location: Azure/azure-sdk/_data/releases/YYYY-MM/<lang>.yml
Monthly YAML files per language, one directory per month:
| Language | Example Path | GitHub URL |
|---|---|---|
| Python | _data/releases/2026-03/python.yml |
Azure/azure-sdk/_data/releases/2026-03/python.yml |
| .NET | _data/releases/2026-02/dotnet.yml |
Azure/azure-sdk/_data/releases/2026-02/dotnet.yml |
| JavaScript | _data/releases/2026-03/js.yml |
Azure/azure-sdk/_data/releases/2026-03/js.yml |
| Java | _data/releases/2026-03/java.yml |
Azure/azure-sdk/_data/releases/2026-03/java.yml |
| Go | _data/releases/2026-03/go.yml |
Azure/azure-sdk/_data/releases/2026-03/go.yml |
| C++ | _data/releases/2026-03/cpp.yml |
Azure/azure-sdk/_data/releases/2026-03/cpp.yml |
Monthly directories exist from 2021-05 to present (new directory created each month automatically).
Each entry contains inline changelog content with standardized section headers. No need to fetch individual CHANGELOG.md files for ~70% of packages.
Example — Feature release (no breaking changes):
# From _data/releases/2026-03/python.yml
- Name: azure-cosmos
Version: 4.15.0
DisplayName: Cosmos DB
ServiceName: Cosmos DB
VersionType: GA
Hidden: false
ChangelogUrl: https://github.com/Azure/azure-sdk-for-python/tree/azure-cosmos_4.15.0/sdk/cosmos/azure-cosmos/CHANGELOG.md
ChangelogContent: |-
#### Features Added
* GA support of Per Partition Automatic Failover and AvailabilityStrategy features.
#### Bugs Fixed
* Fixed bug where sdk was encountering a timeout issue caused by infinite recursion
during the 410 (Gone) error.
* Fixed crash in sync and async clients when `force_refresh_on_startup` was set to `None`.→ Classification: FEATURE (has "Features Added", no "Breaking Changes")
Example — Breaking change release:
# From _data/releases/2026-02/dotnet.yml
- Name: Azure.Compute.Batch
Version: 1.2.0
DisplayName: Batch
ServiceName: Batch
VersionType: GA
Hidden: false
ChangelogUrl: https://github.com/Azure/azure-sdk-for-net/tree/Azure.Compute.Batch_1.2.0/sdk/batch/Azure.Compute.Batch/CHANGELOG.md
ChangelogContent: |-
#### Breaking Changes
- Removed `DataDisk.StorageAccountType`. Use `DataDisk.ManagedDisk.StorageAccountType` instead→ Classification: BREAKING (has "Breaking Changes" section)
Example — Patch/bugfix only release:
- Name: azure-storage-blob
Version: 12.24.1
DisplayName: Storage - Blobs
ServiceName: Storage
VersionType: Patch
ChangelogContent: |-
#### Bugs Fixed
* Fixed an issue where blob name was not being encoded correctly.→ Classification: PATCH (VersionType: Patch, only "Bugs Fixed" section)
Example — Empty ChangelogContent (fallback to CHANGELOG.md needed):
# ~30% of entries, typically azure-mgmt-* packages
- Name: azure-mgmt-cosmosdb
Version: 10.0.0
DisplayName: Resource Management - Cosmos DB
ServiceName: Cosmos DB
VersionType: GA
Hidden: false
ChangelogUrl: https://github.com/Azure/azure-sdk-for-python/tree/.../CHANGELOG.md
ChangelogContent: ""→ Classification: Fetch CHANGELOG.md from ChangelogUrl, or infer from major version bump (10.0.0 = likely breaking)
Fallback source: CHANGELOG.md files (for empty ChangelogContent entries)
Location: Per-package in each SDK repo, e.g.:
Azure/azure-sdk-for-python/sdk/cosmos/azure-cosmos/CHANGELOG.mdAzure/azure-sdk-for-js/sdk/storage/storage-blob/CHANGELOG.md
These use the same standardized section headers as the YAML content:
## 4.9.0 (2024-10-29)
### Features Added
- Added support for the `hierarchical_partition_keys` parameter.
### Breaking Changes
- Renamed `availability_strategy_config` to `availability_strategy`.
### Bugs Fixed
- Fixed crash when `force_refresh_on_startup` was `None`.Do you need CHANGELOG.md at all? Only as a fallback for the ~30% of monthly YAML entries with ChangelogContent: "". For the majority of packages, the YAML provides the full changelog inline.
For each package with a version change detected (Step 1 + 2):
1. Read monthly YAML entry for the package
└─ Parse ChangelogContent field
2. If ChangelogContent is populated:
└─ Has "#### Breaking Changes" section? → BREAKING
└─ Has "#### Features Added" only? → FEATURE
└─ Has "#### Bugs Fixed" only? → PATCH
└─ VersionType == "Patch"? → PATCH
3. If ChangelogContent is empty (""):
└─ Fetch CHANGELOG.md from ChangelogUrl
└─ Parse for same section headers
└─ Apply same logic as step 2
4. Additional CSV signals (catch-all):
└─ Major version bump (1.x → 2.x)? → BREAKING
└─ Support = "deprecated"? → BREAKING (package-level)
└─ Replace column populated? → BREAKING (with migration guide)
5. Result:
└─ BREAKING → flag for human review, update acceptance criteria
└─ FEATURE → auto-update SDK reference, run eval harness
└─ PATCH → auto-update version number only
The CSV files (_data/releases/latest/<lang>-packages.csv) provide version numbers and package metadata but no changelog content. Here's what the CSV gives you:
| CSV Column | What it tells you | Enough for classification? |
|---|---|---|
VersionGA |
Latest stable version number | Can detect major bumps if you snapshot and compare over time — but no history built in |
VersionPreview |
Latest preview version | Same limitation |
Support |
active / deprecated |
Yes — detects deprecated packages |
Replace |
Replacement package name | Yes — detects package-level breaking replacement |
ReplaceGuide |
Migration guide URL | Yes — links to migration docs |
New |
true / false |
Yes — detects new packages |
PlannedVersions |
Upcoming versions with dates | Can infer upcoming major bumps |
The critical gap: The CSV has zero information about what changed in a release. Azure SDK policy says major bump = breaking, but breaking changes can ship in minor versions too (Architecture Board exceptions, preview-to-GA transitions). Real example:
| Scenario | CSV sees | YAML sees |
|---|---|---|
Azure.Compute.Batch 1.2.0 removed DataDisk.StorageAccountType |
VersionGA: 1.1.0 → 1.2.0 — looks like a harmless minor bump |
ChangelogContent has #### Breaking Changes section — caught |
azure-cosmos 4.15.0 added Per Partition Automatic Failover |
VersionGA: 4.14.0 → 4.15.0 — looks identical to above |
ChangelogContent has #### Features Added only — correctly classified as non-breaking |
azure-mgmt-cosmosdb 10.0.0 major rewrite |
VersionGA: 9.x → 10.0.0 — major bump detected |
Same conclusion — but CSV catches this one via semver alone |
CSV-only classifier (~80% accuracy):
Major bump (1.x → 2.x)? → BREAKING ✅ Correct
Support = "deprecated"? → BREAKING ✅ Correct
Replace populated? → BREAKING ✅ Correct
Minor bump (1.1 → 1.2)? → FEATURE ⚠️ Misses breaking changes in minor versions
Patch bump (1.1.1 → 1.1.2)? → PATCH ✅ Usually correct
New = "true"? → NEW PACKAGE ✅ Correct
Note: CSV-only requires periodic snapshots since it only shows the latest version, not version history. You'd need to diff today's CSV against last week's to detect what changed.
Recommendation: Start with CSV-only (Tier 1 — simple, immediate, ~80% accuracy). Add YAML ChangelogContent parsing (Tier 2) to catch the edge cases that matter most — breaking changes hidden in minor versions.
| Tool | Repo | Path | Purpose |
|---|---|---|---|
ChangeLog-Operations.ps1 |
Azure/azure-sdk-tools |
eng/common/scripts/ChangeLog-Operations.ps1 |
Parses CHANGELOG.md section headers |
SemVer.ps1 |
Azure/azure-sdk-tools |
eng/common/scripts/SemVer.ps1 |
Semver parsing, VersionType determination |
version.ts |
Azure/azure-sdk-tools |
tools/js-sdk-release-tools/src/utils/version.ts |
hasBreakingChange → version bump mapping |
changelogGenerator.ts |
Azure/azure-sdk-tools |
tools/js-sdk-release-tools/src/changelog/changelogGenerator.ts |
38 change categories (13 features, 25 breaking) |
Tier 1 — Low effort, high value (immediate):
-
SDK Inventory Script — Scan all
references/sdk/*.mdacross both repos, extract package names via regex (pip install X,npm install X,dotnet add X), producesdk-inventory.json -
Stale SDK Detector — Scheduled GitHub Action that polls the Azure SDK release CSVs (
python-packages.csv,dotnet-packages.csv,js-packages.csv,java-packages.csv,go-packages.csv) for latest versions, compares against inventory, opens issues when drift > 1 minor version -
Cross-Repo Learn URL Checker — Weekly validation that all Microsoft Learn URLs referenced in skills still resolve (404 detection)
Tier 2 — Medium effort (next quarter):
-
AI-Assisted Update Pipeline ("sdk2skills"):
- Trigger: Stale SDK Detector finds new version
- Fetch SDK changelog + release notes from GitHub Releases API
- Use LLM to draft updated
references/sdk/*.mdcontent - Run token checks + eval harness automatically
- Create PR with
[auto-update]label for human review - Estimated: 2-3 weeks to build, saves 2-8 hours per SDK update
-
Shared SDK Inventory — Single source of truth for SDK versions consumed by both
microsoft/skillsandazure-skills, enabling coordinated updates
Tier 3 — High effort, highest value (aspirational):
-
Adversarial Acceptance Criteria Generation:
- When new SDK version detected, run an LLM against the new SDK without the skill
- Capture hallucinated patterns (wrong class names, deprecated methods, incorrect auth)
- Auto-generate acceptance criteria from the failures
- This automates the most valuable artifact in
microsoft/skills
-
Unified Skill Pipeline — Single pipeline that, given an SDK update, produces:
- Updated Learn docs pointers (for
MicrosoftDocs/Agent-Skills) - Updated SDK patterns per language (for
microsoft/skills) - Updated operational references (for
azure-skills)
- Updated Learn docs pointers (for
Yes — this solves the #1 problem: staleness.
The separate-repo model relies on SDK authors voluntarily contributing to a repo they don't own. The co-located model makes skill maintenance part of the natural SDK development workflow.
Today (separate repos):
SDK author ships change in Azure/azure-sdk-for-python
│
▼
SDK author must remember to go update a skill
in a completely separate repo (microsoft/skills)
│
▼
Often nobody does → skill goes stale
With co-location:
SDK author ships change in Azure/azure-sdk-for-python
│
▼
PR reviewer asks: "Did you update the skill?"
(same repo, same PR, same review)
│
▼
Skill stays current by default
| Argument | Why it matters |
|---|---|
| Same PR, same author, same review | The person who renames AgentsClient to PersistentAgentsClient is the exact person who should update the acceptance criteria. Today they'd need a separate PR in a separate repo. Most won't. |
| CI can enforce freshness | SDK repos already validate changelogs and API reviews. Adding "if you modified the public API, update the skill" is trivial in the same repo. Cross-repo enforcement is nearly impossible. |
| Changelog data is right there | The CHANGELOG.md is in the same directory as the skill. No polling, no sync — the PR itself is the trigger. |
| Acceptance criteria are version-specific | Correct/incorrect code patterns are tied to a specific SDK version. Co-location lets them be versioned together. git blame shows which SDK change caused the skill update. |
| SDK teams own their domain | The azure-cosmos team knows what LLMs get wrong about their SDK better than anyone. |
| Argument | Mitigation |
|---|---|
| Fragmentation (6+ SDK repos) | Central aggregation repo (azure-skills) syncs skills from all SDK repos into one install point |
| Consumer discovery degrades | Consumers install from azure-skills, never from SDK repos directly |
| Cross-cutting skills don't fit | Operational skills (deploy, diagnose, cost) stay in azure-skills — only SDK-specific skills co-locate |
| Consistency harder to enforce | Shared eng/common/scripts/ already syncs across all Azure SDK repos — same pattern for skill format validation |
| Eval harness needs a central home | RALPH harness stays in azure-skills, pulls skills from SDK repos to evaluate |
SDK authors write only what they uniquely know — two files alongside their package:
Azure/azure-sdk-for-python/
sdk/cosmos/azure-cosmos/
├── CHANGELOG.md ← already maintain this
├── src/ ← the SDK code
├── tests/ ← SDK unit tests
└── skills/ ← NEW (co-located)
├── SKILL.md ← SDK usage patterns, trigger phrases
└── acceptance-criteria.md ← correct/incorrect code pairs
They do NOT write scenarios.yaml or test harness code. The acceptance-criteria.md IS the test spec. Scenarios are auto-generated centrally.
The SDK author writes acceptance criteria in plain markdown:
### 1. Client Creation
#### Correct
```python
from azure.cosmos import CosmosClient
from azure.identity import DefaultAzureCredential
client = CosmosClient(endpoint, DefaultAzureCredential())
```
#### Incorrect
```python
# WRONG - using master key instead of managed identity
client = CosmosClient(endpoint, master_key)
# WRONG - old import path
from azure.cosmos.cosmos_client import CosmosClient
```
### 2. Parameterized Queries
#### Correct
```python
items = container.query_items(
query="SELECT * FROM c WHERE c.category = @category",
parameters=[{"name": "@category", "value": category}]
)
```
#### Incorrect
```python
# WRONG - SQL injection via f-string
items = container.query_items(
query=f"SELECT * FROM c WHERE c.category = '{category}'"
)
```The central repo auto-generates scenarios.yaml from this:
# AUTO-GENERATED from acceptance-criteria.md
# Do not edit — regenerate via: npm run generate-scenarios azure-cosmos-py
config:
model: claude-sonnet-4-6
max_tokens: 4096
scenarios:
- prompt: "Create a Cosmos DB client in Python"
expected_patterns:
- "DefaultAzureCredential" # from §1 Correct
- "CosmosClient"
forbidden_patterns:
- "master_key" # from §1 Incorrect
- "cosmos_client import" # from §1 Incorrect
- prompt: "Query items from a Cosmos DB container in Python"
expected_patterns:
- "@category" # from §2 Correct
- "parameters"
forbidden_patterns:
- "f\"" # from §2 Incorrect
- "f'" # from §2 IncorrectSDK REPOS (authoring) CENTRAL REPO (aggregation)
┌────────────────────────┐ ┌──────────────────────────┐
│ Azure/azure-sdk-for- │ │ azure-skills │
│ python/ │ │ │
│ sdk/cosmos/ │ automated │ plugin/skills/ │
│ azure-cosmos/ │───sync──────▶│ azure-cosmos/ │
│ CHANGELOG.md │ │ references/ │
│ skills/ │ │ sdk-patterns/ │
│ SKILL.md │ │ python.md ◄── │
│ acceptance- │ │ dotnet.md ◄── │
│ criteria.md │ │ sdk/ │
│ │ │ (install refs) │
│ Azure/azure-sdk-for- │ │ troubleshooting/ │
│ net/ │ │ (from Agent- │
│ sdk/compute/batch/ │───sync──────▶│ Skills) │
│ Azure.Compute.Batch│ │ │
│ CHANGELOG.md │ │ tests/ │
│ skills/ │ │ harness/ (RALPH) │
│ SKILL.md │ │ azure-cosmos/ │
│ acceptance- │ │ scenarios.yaml │
│ criteria.md │ │ (auto-generated) │
│ │ │ │
│ │ │ Platform configs │
│ │ │ (.claude/, .opencode/) │
└────────────────────────┘ └──────────────────────────┘
SDK Author's responsibility Central team's responsibility
(in SDK repo) (in azure-skills)
────────────────────────── ──────────────────────────────
✏️ Write SKILL.md 🔄 Sync skills from SDK repos
✏️ Write acceptance-criteria.md 🤖 Auto-generate scenarios.yaml
✅ Pass format linter (lightweight) 🧪 Run RALPH eval harness
📊 Post results to SDK repo PR
📦 Aggregate into platform configs
🔍 Catalog for discovery/install
The SDK repo only needs a simple markdown linter — no LLM evaluation, no mock responses:
SDK repo pre-commit checks:
✓ acceptance-criteria.md has at least 3 sections
✓ Each section has both "Correct" and "Incorrect" blocks
✓ Code blocks have language tags (```python, ```csharp)
✓ SKILL.md frontmatter is valid (name, description, WHEN triggers)
✓ SKILL.md < 500 lines
The heavy evaluation happens centrally in azure-skills after sync.
Yes — from three repos to a two-repo model with SDK co-location.
Three repos with overlapping scope, inconsistent automation, and no coordination mechanism:
TODAY:
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ MicrosoftDocs/ │ │ microsoft/ │ │ azure-skills │
│ Agent-Skills │ │ skills │ │ │
│ │ │ │ │ │
│ Auto-generated │ │ Manual SDK │ │ Manual ops │
│ doc links │ │ patterns │ │ runbooks │
│ │ │ │ │ │
│ No code │ │ Code patterns │ │ MCP + scripts │
│ │ │ per language │ │ │
│ 193 services │ │ 132 skills │ │ 24 skills │
│ │ │ │ │ │
│ ── No link ──── │──│── No link ───── │──│── Publishes to │
│ │ │ │ │ azure-skills │
└─────────────────┘ └─────────────────┘ └─────────────────┘
▲ ▲ ▲
│ │ │
SDK developer must update all three independently
Phase 1: Co-locate SDK skills (Month 1-2)
- Define the
skills/directory spec for SDK repos (SKILL.md + acceptance-criteria.md format) - Add
skills/directories to top 10 most-used SDK packages, seeded from existing microsoft/skills content - Add lightweight format linter to SDK repo CI (shared via
eng/common/scripts/) - Build the sync automation in azure-skills to pull from SDK repos
Phase 2: Aggregate and evaluate (Month 3-4)
- Sync pipeline pulls skills from all SDK repos into azure-skills
references/sdk-patterns/ - Auto-generate scenarios.yaml from acceptance criteria
- Run RALPH eval harness centrally, post results back to SDK repo PRs
- Integrate Agent-Skills Learn URLs into azure-skills reference files (troubleshooting, security, limits)
- Publish unified catalog with skills from all sources
Phase 3: Deprecate microsoft/skills (Month 5-6)
- All SDK-specific skills now authored in SDK repos, aggregated in azure-skills
- Operational skills remain in azure-skills
- RALPH eval harness lives in azure-skills
- Archive microsoft/skills (read-only) with redirect
- Result: two repos instead of three
FUTURE (two-repo model + SDK co-location):
┌─────────────────────────────────────────────────────────────────┐
│ SDK REPOS (authoring) │
│ Azure/azure-sdk-for-python ─┐ │
│ Azure/azure-sdk-for-net ─┤ Each package has skills/ │
│ Azure/azure-sdk-for-js ─┤ SKILL.md │
│ Azure/azure-sdk-for-java ─┤ acceptance-criteria.md │
│ Azure/azure-sdk-for-go ─┤ │
│ Azure/azure-sdk-for-rust ─┘ Lightweight CI linter │
└──────────────────────┬──────────────────────────────────────────┘
│ automated sync
▼
┌──────────────────────────────────────────────────────────────────┐
│ azure-skills (aggregation + operations + evaluation) │
│ │
│ plugin/skills/ ← Operational skills (hand-crafted) │
│ azure-deploy/ + SDK patterns (synced from │
│ azure-diagnostics/ SDK repos) │
│ azure-cost-optimization/ + Agent-Skills Learn URLs │
│ azure-storage/ (synced from MicrosoftDocs) │
│ references/ │
│ sdk-patterns/python.md ◄── from azure-sdk-for-python │
│ sdk-patterns/dotnet.md ◄── from azure-sdk-for-net │
│ troubleshooting/ ◄── from MicrosoftDocs/Agent- │
│ storage-security.md Skills │
│ │
│ tests/harness/ (RALPH) ← Central eval infrastructure │
│ tests/scenarios/ (auto-gen) ← Generated from acceptance criteria│
│ .claude/ .opencode/ ← Multi-platform configs │
└──────────────────────────────────────────────────────────────────┘
▲ auto-generated Learn URLs
│
┌──────────────────────┴──────────────────────────────────────────┐
│ MicrosoftDocs/Agent-Skills (documentation index) │
│ │
│ 193 services, auto-generated weekly from Microsoft Learn │
│ Troubleshooting, Security, Best Practices, Limits & Quotas │
│ URLs consumed by azure-skills reference files │
└──────────────────────────────────────────────────────────────────┘
| Decision | Recommendation | Rationale |
|---|---|---|
| Should SDK skills live in SDK repos? | Yes — co-locate SKILL.md + acceptance-criteria.md alongside the SDK package | Solves staleness; same PR, same author, same review. Skill updates become part of natural SDK development. |
| Should SDK authors write scenarios.yaml? | No — auto-generate from acceptance criteria centrally | SDK authors write what they know (correct/incorrect code). Central team handles test infrastructure. |
| Should microsoft/skills be retired? | Yes — after Phase 2 migration | SDK skills move to SDK repos; eval harness + operational skills move to azure-skills. microsoft/skills becomes redundant. |
Should azure-skills consume Agent-Skills as fallback? |
Yes — for services without a deep operational skill | Provides baseline coverage for 170+ services via Learn URL references. |
| Should the eval harness live centrally? | Yes — in azure-skills | One harness pulls skills from multiple SDK repos, runs evaluation, posts results back. |
| How does the consumer experience work? | Install from azure-skills only | Consumers never touch SDK repos. azure-skills aggregates everything into one install point with platform configs. |
| Purpose | Path |
|---|---|
| Skills | plugin/skills/ |
| Skill authoring guide | .github/skills/skill-authoring/ |
| Tests | tests/ |
| Skill registry | tests/skills.json |
| Scripts/CLI tools | scripts/src/ |
| Publish pipeline | .github/workflows/publish-to-microsoft-azure-skills.yml |
| PR validation | .github/workflows/pr.yml |
| Purpose | Path |
|---|---|
| Core skills | .github/skills/ |
| SDK plugin skills | .github/plugins/azure-sdk-<language>/skills/ |
| Language symlinks | skills/<language>/ |
| Test scenarios | tests/scenarios/ |
| Eval harness | tests/harness/ |
| Custom agents | Root *.agent.md files |
| Docs site | docs-site/ |
| Purpose | Path |
|---|---|
| All skills | skills/ |
| Pipeline config | github-azure-pipeline.yml |
| Pipeline artifacts | products/ |
microsoft/skills is larger than it appears. Removing it means losing:
| Artifact | Count | Exists elsewhere? | Impact |
|---|---|---|---|
| SDK SKILL.md files | 171 (across 6 languages) | No — azure-skills has 24 ops skills, not SDK skills | Critical — SDK coding guidance disappears |
| Acceptance criteria (correct/incorrect code) | 122 files | No | Critical — the anti-hallucination layer vanishes |
| Test scenarios (scenarios.yaml) | 140 files | No | High — no way to evaluate SDK code quality |
| RALPH eval harness (TypeScript) | ~12 source files | No | High — the only code-quality evaluation infrastructure |
| Custom agents (planner, backend, frontend, etc.) | 6 agent definitions | No | Medium — multi-agent orchestration patterns lost |
| Prompt templates | 5 files | No | Low — convenience, not critical |
| Docs site (Astro) | Full app + GitHub Pages | No | Medium — discovery/catalog gone |
| Multi-platform configs (Claude Code, OpenCode, Context7) | 5 platforms | No | High — only repo supporting non-Copilot agents |
| Hooks system (continual learning) | 1 hook | No | Low |
Bottom line: 171 SDK skills, 122 acceptance criteria, and the eval harness are exclusive. These cannot be recovered from azure-skills or Agent-Skills.
| Content | Why it survives |
|---|---|
| Azure operational skills (deploy, diagnose, cost, RBAC, etc.) | Already in azure-skills — microsoft/skills mirrors them via sync |
| Learn documentation references | In MicrosoftDocs/Agent-Skills (193 services, auto-updated) |
| Mature SDK basics (Storage, Event Hubs patterns) | Largely in LLM pretraining already |
| Cloud architecture patterns (WAF, design patterns) | In LLM pretraining + Agent-Skills |
Yes, but not as a simple folder merge. The repos have fundamentally different structures:
| Dimension | azure-skills (today) | microsoft/skills |
|---|---|---|
| Skill location | plugin/skills/ |
.github/plugins/azure-sdk-<lang>/skills/ |
| Naming | azure-compute, azure-storage |
azure-cosmos-py, azure-service-bus-dotnet |
| Granularity | 1 skill per Azure domain | 1 skill per SDK × language |
| Test framework | Jest + Copilot CLI integration tests | Vitest + RALPH eval harness + scenarios.yaml |
| CI strictness | 4 jobs, 12+ checks, blocking | 2 jobs, non-blocking evals |
| Platform | GitHub Copilot primary | Multi-platform (Copilot + Claude Code + OpenCode) |
Rather than importing 171 separate skills, absorb the SDK content as reference files within existing azure-skills skills:
BEFORE (two repos):
┌─────────────────────────┐ ┌─────────────────────────────────┐
│ azure-skills │ │ microsoft/skills │
│ │ │ │
│ plugin/skills/ │ │ .github/plugins/ │
│ azure-storage/ │ │ azure-sdk-python/skills/ │
│ SKILL.md (ops) │ │ azure-storage-blob-py/ │
│ references/ │ │ SKILL.md (SDK patterns) │
│ sdk/*.md │ │ references/ │
│ │ │ acceptance-criteria.md │
│ azure-messaging/ │ │ azure-sdk-dotnet/skills/ │
│ SKILL.md (ops) │ │ azure-service-bus-dotnet/ │
│ references/ │ │ SKILL.md (SDK patterns) │
│ sdk/*.md │ │ references/ │
│ │ │ acceptance-criteria.md │
└─────────────────────────┘ └─────────────────────────────────┘
AFTER (one repo):
┌──────────────────────────────────────────────────────────┐
│ azure-skills (consolidated) │
│ │
│ plugin/skills/ │
│ azure-storage/ │
│ SKILL.md (ops — unchanged) │
│ references/ │
│ sdk/ │
│ azure-storage-blob-py.md ← existing │
│ azure-storage-blob-ts.md ← existing │
│ azure-storage-blob-java.md ← existing │
│ sdk-patterns/ ← NEW (from ms/skills)│
│ python.md ← SDK usage patterns │
│ dotnet.md ← SDK usage patterns │
│ acceptance-criteria-py.md ← anti-hallucination │
│ acceptance-criteria-dotnet.md │
│ troubleshooting/ ← NEW (from Agent-Skills)│
│ storage-security.md ← Learn URLs │
│ storage-limits.md ← Learn URLs │
│ │
│ tests/ │
│ azure-storage/ │
│ unit.test.ts ← existing │
│ triggers.test.ts ← existing │
│ integration.test.ts ← existing │
│ sdk-eval.test.ts ← NEW: RALPH-style evals │
│ scenarios.yaml ← NEW: from microsoft/skills │
└──────────────────────────────────────────────────────────┘
| microsoft/skills artifact | Where it goes in azure-skills |
|---|---|
| SKILL.md SDK patterns | references/sdk-patterns/<lang>.md per skill |
| acceptance-criteria.md | references/sdk-patterns/acceptance-criteria-<lang>.md |
| scenarios.yaml | tests/<skill>/scenarios.yaml |
| RALPH eval harness | tests/harness/ (new directory, ported from microsoft/skills) |
| Custom agents | .github/agents/ (new directory) |
| Multi-platform configs | .claude/skills/, .opencode/skills/ (new symlinks) |
| Artifact | Why drop it |
|---|---|
| Per-language skill duplication | One azure-storage skill with language-specific reference files, not 5 separate skills |
| Docs site | Not needed — azure-skills already publishes to microsoft/azure-skills |
| Hooks system | Can be added later if needed |
| Prompt templates | Low value, project-specific |
Today, an SDK author updating an Azure SDK must contribute to 2-3 repos:
TODAY (3-repo flow):
1. Update Learn docs → MicrosoftDocs/Agent-Skills auto-crawls
2. Update SDK patterns per language → microsoft/skills PR (×6 languages)
3. Update operational references → azure-skills PR (if applicable)
Steps: 7-15+ files across 2-3 repos
Time: 8-40 hours depending on languages
After consolidation, SDK authors contribute to 1-2 repos:
AFTER (2-repo flow):
1. Update Learn docs → MicrosoftDocs/Agent-Skills auto-crawls
2. Update SDK refs + patterns → azure-skills PR (one repo, one PR)
Steps: 2-5 files in one repo
Time: 2-8 hours
No. The contribution tiers should be explicit:
Who: Any SDK author shipping a version update.
What: Add or update a file in references/sdk/<package-name>-<lang>.md.
Requirements:
- Package name, version, install command
- 3-5 code snippets (auth, CRUD, common operation)
- Bump
metadata.versionin parent SKILL.md (patch increment)
CI gates: Token check + reference integrity only. No test requirement. No scenario thinking needed.
Files touched: 2
1. plugin/skills/azure-storage/references/sdk/azure-storage-blob-py.md (new/updated)
2. plugin/skills/azure-storage/SKILL.md (version bump)
Who: SDK author shipping a new SDK or breaking change.
What: Add references/sdk-patterns/<lang>.md with correct usage patterns, plus acceptance-criteria-<lang>.md with correct/incorrect code pairs.
Requirements:
- Correct code patterns (what LLMs should generate)
- Incorrect code patterns (what LLMs actually hallucinate)
- Test scenarios in
tests/<skill>/scenarios.yaml
CI gates: Token check + reference integrity + RALPH eval (non-blocking initially). Moderate scenario thinking: "What does an LLM get wrong with my SDK?"
Files touched: 3-4
1. references/sdk-patterns/python.md (new/updated)
2. references/sdk-patterns/acceptance-criteria-py.md (new/updated)
3. tests/azure-storage/scenarios.yaml (new/updated)
4. SKILL.md (version bump)
Who: Azure service team launching a new service with deployment/diagnostics needs. What: Create a complete new skill with SKILL.md, references, tests, and integration test registration. Requirements:
- Full skill authoring guide compliance
- Trigger/non-trigger test prompts (5+ each)
- Integration test scenarios
- Registration in
tests/skills.json
CI gates: All 4 job categories, blocking. Deep scenario thinking required.
Files touched: 6-10+
Full skill directory + test directory + skills.json registration
| Dimension | Today (microsoft/skills) | Today (azure-skills) | After consolidation |
|---|---|---|---|
| Repos to PR | 1 (microsoft/skills) | 1 (azure-skills) | 1 (azure-skills) |
| Files for SDK ref | 2-4 per language | 2-3 | 2 (Tier 1) |
| Test requirement | scenarios.yaml (encouraged) | 3 test files (encouraged) | None for Tier 1 |
| CI strictness | Non-blocking evals | 4 blocking jobs | Token + refs only for Tier 1 |
| Version bump | Not enforced | Mandatory | Mandatory (auto-increment option) |
| Skill registration | Not required | Required in skills.json | Required only for Tier 3 |
| Time to contribute | 2-8 hours/language | 4-16 hours | 30 min (Tier 1) to 16 hours (Tier 3) |
- Port the RALPH eval harness into
azure-skills/tests/harness/ - Import acceptance criteria for the top 10 most-used SDKs as
references/sdk-patterns/acceptance-criteria-<lang>.mdfiles in existing skills - Import test scenarios as
tests/<skill>/scenarios.yaml - Add multi-platform configs (
.claude/skills/,.opencode/skills/symlinks)
- For each azure-skills skill that has SDK references, merge microsoft/skills SKILL.md content into
references/sdk-patterns/<lang>.md - For SDK skills with no matching azure-skills skill, create thin operational wrappers (just SKILL.md + SDK pattern references, no deep ops content)
- Update CI to support tiered validation (Tier 1/2/3 based on which files changed)
- Archive microsoft/skills (read-only)
- Update README with redirect to azure-skills
- Update marketplace.json and platform configs to point to azure-skills
- Announce deprecation via GitHub release + docs site banner
| Risk | Mitigation |
|---|---|
| Community backlash (1,693 stars) | 3-month deprecation window with clear redirect |
| Multi-platform support gaps | Port all platform configs in Phase 1 before deprecation |
| Eval harness incompatibility | Run both test frameworks in parallel during Phase 2 |
| SDK author confusion | Clear Tier 1/2/3 documentation + CLI scaffolding tool |
| Token budget overflow | SDK patterns as JIT references, not in main SKILL.md |
| Question | If you remove microsoft/skills | If you keep it |
|---|---|---|
| SDK author experience | Better — one repo, tiered contributions, 30-min path for simple updates | Unchanged — 2-3 repos, 8-40 hours |
| Maintenance burden | Better — one repo to maintain, one CI system | Two repos, two CI systems, no sync mechanism |
| Anti-hallucination coverage | Same — acceptance criteria imported into azure-skills | Same — lives in microsoft/skills |
| Multi-platform reach | Same if ported — add Claude Code/OpenCode configs to azure-skills | Better today — already configured |
| Community/discoverability | Worse initially — 1,693 stars don't transfer | Better — separate repos can have separate communities |
| Eval infrastructure | Same if ported — RALPH harness works anywhere | Same — lives in microsoft/skills |
| SDK × Language coverage | Compressed — 6 language files per skill vs 6 separate skills | Expanded — 171 dedicated skills |