maui: Skill for Claude Code

Rebalance UI Test Categories

Use this skill when a category in maui-pr-uitests needs multiple ordinary matrix legs and each leg must fit a target duration.

Rules

• Historical Azure test results are required. Never substitute source test counts for timing evidence.
• Use one method-level [ShardedTestCategory(UITestCategories.Category, shard: N)] attribute. It exposes both the umbrella and exactly one numbered shard category to NUnit. Never put sharded categories on a class.
• Use a conservative per-test/platform p80 of recent valid run totals.
• Aggregate measured fixed job overhead with nearest-rank p80.
• Reserve a configurable safety margin (2 minutes by default), so every projection is strictly below TargetMinutes - SafetyMarginMinutes.
• Assignment is deterministic multidimensional longest-processing-time scheduling over individual methods and cohesive ordered fixtures. It minimizes the worst projected platform/shard duration.
• Fail if one test plus overhead exceeds the target or if -MaxShards cannot satisfy the target.
• Applying a category maintains the analyzer's reusable shard-prefix set, preserving previously registered umbrellas, and performs mandatory plan-to-source validation after editing.
• Before splitting methods from one fixture across shards, ensure fixture setup independently reaches the state required by every selected method. Never rely on an Order(1) test to initialize the remaining shard.
• Dedicated configuration stages that use testConfigurationArgs, such as ios_ui_tests_mono_cv1, reuse the numbered shard filters but remain outside ordinary category-matrix projection data. Their samples and overhead are recorded separately so they cannot make ordinary shard projections look artificially cheaper.

Commands

Gather, plan, report, and apply in one operation:

pwsh .github/skills/rebalance-ui-test-categories/scripts/Rebalance-UITestCategories.ps1 `
  -Mode All -Category CollectionView -TargetMinutes 60 `
  -SafetyMarginMinutes 2 -BuildId 1561395,1563447 `
  -UnmeasuredTestPolicy ClassPlatformMax `
  -OutputPath artifacts/CollectionView-rebalance.json -Apply