Skip to content

Fix #533: guard out_proj.weight access in GDN out-projection ESIMD probes - #557

Open
joaovgaraujo wants to merge 1 commit into
intel:mainfrom
joaovgaraujo:fix/gdn-outproj-weight-getattr-533
Open

Fix #533: guard out_proj.weight access in GDN out-projection ESIMD probes#557
joaovgaraujo wants to merge 1 commit into
intel:mainfrom
joaovgaraujo:fix/gdn-outproj-weight-getattr-533

Conversation

@joaovgaraujo

Copy link
Copy Markdown

Problem

Both GDN out-projection eligibility probes read self.out_proj.weight unconditionally. Quantized RowParallelLinear layers expose .qweight, not .weight, so hybrid GDN+MoE models loaded with GPTQ/AWQ crash on the first forward pass with AttributeError: 'RowParallelLinear' object has no attribute 'weight'.

Fix

w = getattr(self.out_proj, "weight", None) in both _gdn_outproj_esimd_eligible and _gdn_outproj_batched_esimd_eligible (in gdn_linear_attn.py, applied through vllm/patches/vllm_for_multi_arc.patch).

The existing check ok = (w is not None and ...) already short-circuits: when w is None, ok becomes False and execution falls through to the generic self.out_proj(core_attn_out) call, which already dispatches GPTQ/AWQ correctly through int4_gemm_w4a16. For fp8 layers, getattr returns the same tensor object as the direct attribute access, so that path is unchanged.

Testing

Ran on an Intel Arc Pro B70 (32GB, Battlemage G21) with intel/llm-scaler-vllm:0.21.0-b1.

Before: palmfuture/Qwen3.6-35B-A3B-GPTQ-Int4 loads and starts the server, then crashes on the first inference request with the #533 AttributeError.

After: the same model loads, serves, and generates correct output end to end at 15.9 tok/s (single-request decode, GPTQ path). Re-checked the sym_int4 and fp8 out-projection paths on the same card afterward; both still pass.

Closes #533.

…MD probes

Quantized RowParallelLinear layers expose qweight, not weight. Both
GDN out-projection eligibility probes read self.out_proj.weight
unconditionally, so hybrid GPTQ/AWQ models crash with AttributeError
on the first forward pass. getattr with a None default lets the
existing short-circuit (ok = w is not None and ...) fall through to
the generic out_proj path, which already handles GPTQ/AWQ correctly.

Signed-off-by: Joao Vitor Guedes de Araujo <184870811+joaovgaraujo@users.noreply.github.com>
@joaovgaraujo
joaovgaraujo force-pushed the fix/gdn-outproj-weight-getattr-533 branch from 4a14064 to 2ac4ba3 Compare July 22, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant