Borrowing it
Nothing to install: this file belongs to maoxx241/vllm-ascend-workspace. Take a copy, put it at the same path in your own repository, and replace the rules that are about this project with yours.
curl -O https://raw.githubusercontent.com/maoxx241/vllm-ascend-workspace/main/.agents/skills/vllm-ascend-graph-debug/SKILL.mdgit clone --depth 1 https://github.com/maoxx241/vllm-ascend-workspaceWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/maoxx241/vllm-ascend-workspace/vllm-ascend-graph-debug)<a href="https://agentmods.dev/skills/maoxx241/vllm-ascend-workspace/vllm-ascend-graph-debug"><img src="https://agentmods.dev/badge/skills/maoxx241/vllm-ascend-workspace/vllm-ascend-graph-debug/github.svg" alt="Measured on agentmods" height="20"></a>Or the 80×15 button, for a site that already has a row of RSS and ATOM ones. Only the verdict fits; the numbers stay here.
<a href="https://agentmods.dev/skills/maoxx241/vllm-ascend-workspace/vllm-ascend-graph-debug"><img src="https://agentmods.dev/badge/skills/maoxx241/vllm-ascend-workspace/vllm-ascend-graph-debug.svg" alt="Reviewed on agentmods" width="80" height="20"></a>- NVIDIA SkillSpector pass
What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00097 | $0.03088 |
| Opus 5 | $0.00048 | $0.01544 |
| Sonnet 5 | $0.00019 | $0.00618 |
| Haiku 4.5 | $0.00010 | $0.00309 |
Grade A, and why
vllm-ascend-graph-debug scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 11d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 234 lines — stays where its author put it; the contents beside it link to each section on GitHub.
NPU Graph Debug
用于排查 vLLM Ascend 图模式下的编译、捕获、重放和精度问题。核心方法是:记录已知信息,先定性问题阶段,再控制变量缩小范围;只有范围足够小时,才插入预分配 buffer,通过图内 copy_ 和图外落盘对比 graph/eager 的中间状态。
结构化入口
从仓库根目录使用 scripts/graph_debug_case.py:
init创建.vaws-local/graph-debug/<case-id>/case.json和 Run Manifest v1。- 每轮单变量实验后立即用
record追加假设、预期、观测、结论和下一步。 - 需要中间状态对拍时,用
compare对齐 eager/graph JSONL snapshot 并找到首个分叉。 - 修复后用
finalize同时记录最小复现、原始复现和 instrumentation 清理状态。
按需读取:
- Behavior contract:case 生命周期、snapshot schema、比较语义和 Run Manifest 集成。
- Command recipes:可复制的 init、record、compare、finalize 命令。
- Acceptance:结案前必须逐项满足的验收条件。
工作原则
- 每轮实验只改变一个变量,实验前写明假设,实验后记录结论。
- 优先证明“已经排除什么”和“问题首次出现在哪里”,避免重复回到已验证路径。
- Eager 模式必须先正常;如果 Eager 本身失败,先修 Eager,不进入图模式排查。
- 精度排查优先固定随机性和输入;性能、吞吐、并发压力只在问题需要时引入。
- 图内只做设备侧、可 capture 的操作;同步、CPU 读回、文件写入统一放到图外。
记录要求
不要使用散落的临时笔记作为唯一事实来源。以 case 目录中的 case.json 为结构化记录;issue 或 PR comment 可以引用它的摘要。每次实验完成后立即运行 record,并保持“已知事实 / 已排除 / 当前嫌疑 / 下一步”与 case 内容一致。
总流程
- 建立基线:同输入、同 seed、同采样参数分别跑 Eager 和 Graph。
- 固定确定性:启动环境设置
HCCL_DETERMINISTIC=true;model runner 初始化时调用torch.use_deterministic_algorithms(True)。 - 阶段定性:先判断问题发生在 compile、capture、replay 还是 accuracy。
- 控制变量缩小范围:模型规模、并行策略、请求 shape、capture size、特性开关、部署方式逐项收敛。
- 静态审计:检查 graph replay 依赖的 metadata、padding、dummy run 与真实请求路径、自定义算子 capture/replay 约束。
- 动态打点:在候选模块插入预分配 debug buffer,图内
copy_,图外 flush。 - 对齐比较:按 step/layer/rank/tag 先比统计量,再比局部样本,定位首个分叉点。
- 修复验证:关闭 debug 代码和同步逻辑,重跑最小复现和原始复现,更新记录。
阶段定性
| 阶段 | 典型现象 | 先问什么 | 下一步 |
|---|---|---|---|
| Compile | 启动、加载或 torch.compile 阶段报错/卡住 |
是否是 Python/Dynamo/FX/算子前端问题 | 看堆栈;缩小到具体模块或算子;必要时图外打印 |
| Capture | dummy run、capture model 阶段报错/卡住 | 编译后静态图是否能在固定 shape 下执行 | 隔离 compile backend 与 graph capture;查同步、CPU 读回、自定义算子 |
| Replay | capture 成功,真实请求时报错/卡住 | 真实请求和捕获 shape/metadata/通信序列是否一致 | 查 padding、固定地址 metadata、event 配对、rank 间状态 |
| Accuracy | graph 输出与 Eager 非预期差异 | 是良性数值差异还是功能性错误 | 无 padding 对照;静态审计;中间 tensor 对比 |
What ships with it
6 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 11d ago First seen · 234 lines · 97 tokens per session scan A 88bb6088bfc5
vllm-ascend-graph-debug is a skill published in the GitHub repository maoxx241/vllm-ascend-workspace (36 stars, last pushed 6d ago), licensed MIT. It adds 97 tokens to every session and 3,088 once invoked, about $0.0005 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
gke-ai-troubleshooting-tpu-dynamic-slices-monitoring
Monitors, troubleshoots, and manages GKE TPU Dynamic Slices custom resources. Use when checking TPU slice lifecycle states, troubleshooting slice provisioning failures, validating single-slice or multi-slice (JobSet) workload manifests, or safely patching stuck finalizers and disabling the slice controller. Don't use…
gke-ai-troubleshooting-tpu-vbar-oom
Diagnoses and prevents vbarcontrolagent segfaults, out-of-memory (OOM) errors, and TPU device initialization failures on TPU v6e nodes in GKE caused by race conditions during TPU device resets or high-frequency metrics polling. Use when troubleshooting vbarcontrolagent crashes, memory cgroup OOMs in serial console…
competition-firmware-layout
Internal downstream skill for ctf-sandbox-orchestrator. CTF-sandbox workflow for firmware images, partition tables, boot chains, update packages, extracted filesystems, embedded configs, and device-facing trust boundaries. Use when the user asks to unpack firmware, map partition layout, inspect bootloader or init…
doca-flow
Build and debug DOCA Flow applications on supported NVIDIA NICs/DPUs: define match/action pipes, initialize ports and representors, choose forwarding targets, validate pipes before hardware programming, read counters, match the Flow version to the installed DOCA release, and diagnose Flow API errors. Trigger on DOCA…
diagnose-driver-install
Diagnose NVIDIA driver installation failures on DeepOps-managed nodes — nvidia-smi errors, "No devices were found", DKMS build failures, or GPU pods crash-looping. Use before reinstalling anything.
catc-troubleshoot
Catalyst Center troubleshooting workflows - device unreachable investigation, client connectivity issues, interface down analysis, site-wide outage triage, wireless roaming problems, integration with pyATS for CLI-level diagnostics. Use when a device is unreachable, a user reports connectivity problems, an interface…