neuron-framework-autoport-vllm-neuron

vLLM-Neuron Model Porting Skill

Port the HuggingFace model $1 to the vLLM-Neuron Trainium2 backend as $0.

Conventions

• MODEL_NAME = $0 (snake_case, e.g., yi, deepseek_v2)
• HF_MODEL_ID = $1 (e.g., 01-ai/Yi-6B-Chat)
• --review flag (if present in $ARGUMENTS): pause at Step 2 for user confirmation
• Derive PascalName from MODEL_NAME (e.g., yi → Yi, gpt_neox → GPTNeoX)
• Model code goes in vllm_neuron/model
• Example script goes in examples/vllm_neuron/models/MODEL_NAME/

Dry-run

When the user specifies dry-run:

• Skip all agent-level prerequisites (package import checks, NeuronCore check via neuron-ls)
• Activate the provided venv and resolve source paths by filesystem lookup (do NOT use import — dependencies are not installed):

  export PATH=<pathToVenv>/bin:$PATH
  VLLM_NEURON_SRC=~/tmp/vllm_autoport/vllm_neuron
  TRANSFORMERS_SRC=$(python3 -c "import site; print(site.getsitepackages()[0])")/transformers
  

• Do not run any code — no compilation, inference, or validation (no Trainium hardware available)

Before You Start

Ensure the agent-level prerequisites have been completed (venv activation, package verification, NeuronCore check). These steps are owned by the agent prompt and should not be re-executed here.

Note: Do NOT clear /var/tmp/neuron-compile-cache as a pre-flight step — it is a shared system directory and other processes or users may depend on it. Only clear it reactively if you hit a [NLA001] JSON parse error or FileNotFoundError on neff_output paths.

Step 0: Parse & Detect State

1. Package check: Ensure that private_vllm_neuron or vllm_neuron package exists in the user's workspace. If it does not, STOP and do not proceed without informing the user.
2. Parse $ARGUMENTS — extract model-name, hf-model-id, and --review flag.
3. Check if vllm_neuron/model/MODEL_NAME/ already exists:
   • If model.py + registry entry both exist → skip to Phase C (validation)
   • If model.py exists but not registered → resume at Step 5 (register)
   • If directory doesn't exist → proceed to Phase A