feat: add GLM-5.3-Flash MTP support - #2157
Open
cubezhang wants to merge 2 commits into
Open
Conversation
Contributor
🏷️ CI GuideRuns automatically on every eligible PR before approval:
Heavy model tests:
|
Contributor
|
Please fix conflicts |
Contributor
Author
|
Okay,let me check. |
cubezhang
force-pushed
the
glm53-flash-mtp
branch
from
September 8, 2026 02:11
a08ea2f to
196b74f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: add GLM-5.3-Flash MTP support
GLM-5.3-Flash checkpoint layer 45 was previously discarded, so the model could
only serve without speculative decoding. This change registers a GLM-specific
MTP model, reuses the existing NextN predictor prologue, and constructs layer 45
with GLM's k-pool MLA and clamped SwiGLU MoE behavior.
The pooled indexer now preserves position-addressed key/gate history across
speculative verification and rejection, including ragged concurrent requests.
Draft graph staging handles GLM-5.3's two-dimensional NextN residual even though
the backbone config contains
hc_mult=4. The existing GLM-5.2/DeepSeek MTP pathkeeps its original decoder behavior through the default predictor-layer hook.
Validation
top-k sharing, decode contracts, scheduling, GLM routing, and k-pool behavior.
upstream/mainplus this commitcompleted real-model TP8 MTP3 text serving and the 8/8 quality smoke suite on
8× MI308X.
short/long requests were validated during bring-up.
Measured performance
On a fixed natural-language workload with TP8, 512 output tokens, and BF16
KV/index cache:
This branch is based directly on
upstream/main(5a9c206).