ado-sync-judge skill

Overview

This skill provides an LLM-as-Judge for verifying Azure DevOps synchronization correctness and conflict resolution. It evaluates bidirectional sync behavior, enforces lifecycle rules, and audits decisions to ensure external tool status wins and completion counters increment strictly. The tool is implemented in TypeScript and designed for automated validation and human-readable audit outputs.

How this skill works

The judge inspects sync events, compares source and target states, and identifies discrepancies or rule violations. It checks conflict resolution outcomes against a policy that prioritizes external tool status and validates strict increment semantics for completion fields. Results include pass/fail decisions, detailed rationales, and suggested corrective actions.

When to use it

• Validating bidirectional sync between Azure DevOps and an external system before deployment.
• Auditing conflict resolution decisions after a sync run to ensure policy compliance.
• Testing lifecycle management rules for work items across integrated tools.
• Verifying that external tool updates take precedence over internal changes.
• Ensuring completion counters or version fields increment only in allowed scenarios.

Best practices

• Run the judge against representative data sets that include edge cases and concurrent updates.
• Keep sync policies explicit and codified so the judge can evaluate them deterministically.
• Log raw events and the judge’s rationale together to enable fast root-cause analysis.
• Use the judge as part of CI to catch regressions in sync logic before production runs.
• Treat judge findings as actionable: automate common fixes and escalate ambiguous cases.

Example use cases

• Pre-release validation: confirm that changes from a third-party tracker win when conflicts occur.
• Regression testing: detect when a code change causes improper completion increments.
• Audit reports: produce human-readable summaries of sync decisions for compliance reviews.
• Incident investigation: analyze recent sync events to find where lifecycle rules were violated.
• Policy enforcement: block incorrect sync runs by failing CI when judge detects rule breaches.

FAQ