Add Laguna S 2.1 model support

.agents/skills/gptqmodel-tokenizer-normalization/SKILL.md

@@ -0,0 +1,39 @@
+---
+name: gptqmodel-tokenizer-normalization
+description: Diagnose and correct GPT-QModel tokenizer initialization, tokenization normalization, special-token handling, prompt rendering, and chat-template problems. Use when inference or evaluation quality suggests wrong token IDs or prompts, when a model needs tokenizer load kwargs or compatibility patches, or when tokenizer behavior is being changed in GPT-QModel; reusable corrective behavior must be implemented, tested, versioned, pushed, and submitted upstream to github.com/ModelCloud/Tokenicer.
+---
+
+# GPT-QModel tokenizer normalization
+
+Treat [ModelCloud/Tokenicer](https://github.com/ModelCloud/Tokenicer) as the source of truth for reusable tokenizer loading and normalization. GPT-QModel depends on Tokenicer; model loaders should consume its behavior rather than accumulate model-specific tokenizer patches.
+
+## Diagnose the boundary
+
+1. Reproduce with the non-quantized model before attributing low scores or malformed output to quantization.
+2. Capture the exact raw text, rendered chat-template text, input IDs, attention mask, special-token settings, tokenizer class, tokenizer config, and Transformers/Tokenicer versions.
+3. Compare direct `AutoTokenizer` behavior, current `Tokenicer.load()`, and the proposed normalization on the same prompts. Include strings that expose the suspected boundary or regex error.
+4. Run a small dense-model evaluation with the same task, generation settings, and evaluator used for the quantized model. If dense and quantized runs fail similarly, fix the tokenizer or template before changing quantization code.
+5. Separate reusable tokenizer behavior from evaluator-specific prompt construction. AutoTokenizer kwargs, tokenizer compatibility, special-token repair, and model chat-template normalization belong in Tokenicer. A benchmark task's own few-shot or answer-extraction format remains in the evaluation project.
+
+Do not suppress a correctness warning or hard-code a GPT-QModel model-name check when Tokenicer can normalize the underlying behavior. Preserve explicit caller overrides with patterns such as `setdefault` unless the upstream API documents a mandatory correction.
+
+## Implement upstream first
+
+1. Use an existing `/root/tokenicer` clone, or clone `https://github.com/ModelCloud/Tokenicer` when it is absent.
+2. Sync Tokenicer's default branch and create a focused branch. Preserve unrelated worktree files.
+3. Implement the smallest general normalization in Tokenicer and make every tokenizer load and fallback path apply it consistently.
+4. Add focused unit tests for the default correction, explicit override behavior, and fallback paths. Add a lightweight end-to-end tokenizer assertion when it materially strengthens the regression.
+5. Increment the Tokenicer version for the release-bound correction and document the change where that repository's conventions require it.
+6. Run the focused tests and the complete Tokenicer test suite. Resolve every failure before publishing; report genuine environment skips separately.
+7. Review and stage only intended files, commit them, push the branch, and open a pull request against `ModelCloud/Tokenicer`. Include the root cause, before/after tokenization evidence, tests, and downstream GPT-QModel impact.
+
+Do not consider a reusable normalization fixed when it exists only as a GPT-QModel workaround. If temporary downstream code is unavoidable while an upstream release is pending, isolate and label it, link the Tokenicer pull request, and remove it as soon as the dependency can be raised.
+
+## Consume the correction in GPT-QModel
+
+1. Route all normal tokenizer construction through `Tokenicer.load()`; avoid new direct `AutoTokenizer.from_pretrained()` paths in model loaders.
+2. After the Tokenicer release is available, raise GPT-QModel's minimum Tokenicer version and remove duplicated compatibility code in the same change.
+3. Add a GPT-QModel regression that proves the public load path receives normalized behavior without reimplementing Tokenicer internals.
+4. Re-run the dense baseline, then quantized save/reload/inference and the affected evaluation tasks. Compare exact prompt rendering and input IDs before comparing scores.
+
+In the final report, link the Tokenicer commit and pull request, state the Tokenicer version, list all Tokenicer and GPT-QModel checks, and distinguish passed tests from skips or unavailable hardware.

.agents/skills/gptqmodel-tokenizer-normalization/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "GPT-QModel Tokenizer Normalization"
+  short_description: "Route tokenizer and chat-template fixes upstream"
+  default_prompt: "Use $gptqmodel-tokenizer-normalization to diagnose and upstream this tokenizer or chat-template correction."

AGENTS.md

@@ -0,0 +1,11 @@
+# GPT-QModel agent guidance
+
+This file governs the whole repository in addition to any parent agent instructions.
+
+## Tokenizer normalization ownership
+
+For tokenizer initialization, tokenization normalization, special-token compatibility, prompt rendering, chat-template correctness, or unexpectedly low inference/evaluation scores, read and follow `$gptqmodel-tokenizer-normalization` in `.agents/skills/gptqmodel-tokenizer-normalization/SKILL.md` before editing.
+
+GPT-QModel depends on [ModelCloud/Tokenicer](https://github.com/ModelCloud/Tokenicer). Put reusable corrective tokenizer and chat-template normalization in Tokenicer, add relevant tests, increment its version, run its complete test suite, then commit, push, and open a Tokenicer pull request. Do not leave model-loader patches in GPT-QModel merely to avoid fixing the dependency.
+
+Establish a non-quantized baseline before treating bad generation or evaluation scores as a quantization regression. Compare exact rendered prompts and input IDs as well as aggregate scores.

README.md

@@ -21,6 +21,7 @@
 
 ## Latest News
 
+* 07/22/2026 7.3.1 `main`: ✨ Added Poolside `Laguna S 2.1` model support
 * 07/14/2026 7.3.0-dev `main`: ✨ Added `nemotron_h_puzzle` model support
 * 07/07/2026 7.3.0-dev `main`: ✨ Added `deepseek_vl` model support
 * 07/06/2026 7.3.0-dev `main`: ✨ Added `deepseek_ocr2` model support

gptqmodel/models/definitions/laguna.py

@@ -34,7 +34,10 @@ class LagunaQModel(BaseQModel):
             ),
             "post_attention_layernorm": ("post_attention_layernorm:!",),
             "mlp:moe": {
+                # Native Transformers uses the plural name, while Laguna S 2.1's
+                # custom model code keeps the checkpoint's singular name.
                 "shared_experts": ("gate_proj:0", "up_proj:0", "down_proj:1"),
+                "shared_expert": ("gate_proj:0", "up_proj:0", "down_proj:1"),
                 "gate": ("gate:!",),
                 "experts": {
                     "#": ("gate_proj:0", "up_proj:0", "down_proj:1"),

gptqmodel/utils/hf.py

@@ -21,11 +21,9 @@
     AutoConfig,
     AutoModel,
     AutoModelForCausalLM,
-    AutoTokenizer,
     GenerationConfig,
     PreTrainedModel,
 )
-from transformers.models.auto.tokenization_auto import get_tokenizer_config, tokenizer_class_from_name
 
 from ..nn_modules.qlinear.gguf import (
     PRISM_Q1_0_G128_BLOCK_SIZE,
@@ -1770,99 +1768,19 @@ def load_hf_tokenizer(
         trust_remote_code: bool = False,
         **kwargs,
 ):
-    auto_tokenizer_exc = None
-    try:
-        # Preferred path: let transformers perform its normal tokenizer
-        # resolution. This keeps behavior identical for native tokenizers and
-        # for remote-code tokenizers that are still compatible with the
-        # installed transformers release.
-        return AutoTokenizer.from_pretrained(
-            tokenizer_or_path,
-            trust_remote_code=trust_remote_code,
-            **kwargs,
-        )
-    except ValueError as exc:
-        # Transformers 5.x can incorrectly route some legacy/local repos to the
-        # generic tokenizers backend, which then fails before consulting the
-        # declared tokenizer class from tokenizer_config.json.
-        #
-        # In that failure mode the repo may still have a complete
-        # `vocab.json`/`merges.txt` pair, but `AutoTokenizer` never reaches the
-        # model-specific tokenizer class that knows how to map those files to
-        # `vocab_file` / `merges_file`. The issue is therefore dispatch, not
-        # missing tokenizer assets.
-        if "Couldn't instantiate the backend tokenizer" not in str(exc):
-            raise
-        auto_tokenizer_exc = exc
-    except AttributeError as exc:
-        # Narrow fallback for legacy trust_remote_code repositories. On
-        # transformers 5.x, some old repos no longer resolve to a tokenizer
-        # class inside AutoTokenizer and instead fail with
-        # `None.from_pretrained(...)`. Only intercept that specific compat
-        # break; all other exceptions should propagate unchanged.
-        if not trust_remote_code or "from_pretrained" not in str(exc):
-            raise
-        auto_tokenizer_exc = exc
-
-    tokenizer_config = get_tokenizer_config(
-        tokenizer_or_path,
-        trust_remote_code=trust_remote_code,
-        **kwargs,
-    )
-    tokenizer_class_name = tokenizer_config.get("tokenizer_class")
-    if isinstance(tokenizer_class_name, str):
-        # `tokenizer_config.json` is the most direct source of truth once the
-        # generic auto-dispatch path has proven unreliable. Resolving the class
-        # name explicitly lets us instantiate the tokenizer implementation that
-        # ships with transformers (for example `Qwen2Tokenizer`) so it can load
-        # its expected files from the repo in the normal way.
-        tokenizer_cls = tokenizer_class_from_name(tokenizer_class_name)
-        if tokenizer_cls is not None:
-            return tokenizer_cls.from_pretrained(
-                tokenizer_or_path,
-                trust_remote_code=trust_remote_code,
-                **kwargs,
-            )
-
-    auto_map = getattr(model_config, "auto_map", None) or {}
-    # Old repositories often still expose an authoritative dynamic tokenizer
-    # reference in `config.auto_map`, even when the higher-level
-    # AutoTokenizer registry no longer reaches it.
-    class_ref = auto_map.get("AutoTokenizer")
-    if isinstance(class_ref, (list, tuple)):
-        # HF stores tokenizer refs as [slow, fast]. Prefer the fast tokenizer
-        # when present, otherwise use the slow one.
-        class_ref = class_ref[1] if len(class_ref) > 1 and class_ref[1] is not None else class_ref[0]
+    from tokenicer import Tokenicer
 
-    if not isinstance(class_ref, str):
-        raise auto_tokenizer_exc
-
-    from transformers.dynamic_module_utils import get_class_from_dynamic_module
-
-    tokenizer_cls = get_class_from_dynamic_module(class_ref, str(tokenizer_or_path), **kwargs)
-    original_init = getattr(tokenizer_cls, "__init__", None)
-    if callable(original_init) and not getattr(tokenizer_cls, "_gptqmodel_legacy_init_compat", False):
-        def patched_init(self, *init_args, **init_kwargs):
-            # Some legacy tokenizers assign `bos/eos/pad/..._token_id` before
-            # they call `PreTrainedTokenizer.__init__()`. In transformers 5.x
-            # those assignments now go through base-class attribute handling,
-            # which expects `_special_tokens_map` to already exist. Creating
-            # the storage eagerly preserves the old constructor order without
-            # modifying the upstream repository code.
-            if not hasattr(self, "_special_tokens_map"):
-                object.__setattr__(self, "_special_tokens_map", {})
-            return original_init(self, *init_args, **init_kwargs)
-
-        tokenizer_cls.__init__ = patched_init
-        # Avoid wrapping the same dynamically imported class multiple times in
-        # a long-running process.
-        tokenizer_cls._gptqmodel_legacy_init_compat = True
-    tokenizer_cls.register_for_auto_class()
-    return tokenizer_cls.from_pretrained(
+    tokenicer = Tokenicer.load(
         tokenizer_or_path,
+        model_config=model_config,
         trust_remote_code=trust_remote_code,
         **kwargs,
     )
+    # BaseQModel applies GPTQModel's runtime Tokenicer wrapper once the model is
+    # constructed. Keep this loader's native-tokenizer return contract while
+    # delegating tokenizer construction and compatibility normalization to
+    # Tokenicer.
+    return tokenicer.tokenizer
 
 
 

requirements.txt

@@ -10,7 +10,7 @@ device-smi>=0.5.5
 protobuf>=7.34.0
 pillow>=11.3.0
 pypcre>=0.3.2
-tokenicer>=0.0.13
+tokenicer>=0.0.14
 logbar>=0.4.3
 jinja2>=3.1.0
 ninja>=1.13.0

tests/models/test_laguna.py

@@ -3,6 +3,7 @@
 # SPDX-License-Identifier: Apache-2.0
 # Contact: qubitium@modelcloud.ai, x.com/qubitium
 
+import copy
 import os
 
 from model_test import ModelTest
@@ -11,8 +12,8 @@
 from gptqmodel.utils.backend import BACKEND
 
 
-class TestLagunaXS2(ModelTest):
-    NATIVE_MODEL_ID = os.environ.get("GPTQMODEL_LAGUNA_XS2_MODEL", "/monster/data/model/Laguna-XS.2")
+class _LagunaModelTest(ModelTest):
+    """Share Laguna quantization coverage across checkpoints with the same MoE topology."""
 
     FALLBACK = Fallback()
     LOAD_BACKEND = BACKEND.AUTO
@@ -24,6 +25,7 @@ class TestLagunaXS2(ModelTest):
     # three layers keeps the true-model path fast while covering two split MoE layers.
     MODEL_COMPAT_FAST_LAYER_COUNT = 3
     MODEL_COMPAT_FAST_LAYER_POSITION = "first"
+    MIN_FAST_MOE_LAYER_COUNT = 2
     DATASET_SIZE_FAST = 128
     DATASET_CONCAT_SIZE_FAST = 1024
     EVAL_BATCH_SIZE = "auto"
@@ -100,10 +102,43 @@ def _assert_fast_quant_covers_defused_moe_layers(self, model):
             if has_split_expert:
                 covered.append(layer_idx)
 
-        assert len(covered) >= 2, (
-            "Laguna fast quant test must cover at least two split MoE expert layers; "
+        assert len(covered) >= self.MIN_FAST_MOE_LAYER_COUNT, (
+            "Laguna fast quant test did not cover enough split MoE expert layers; "
             f"covered={covered}, selected={list(selected_indexes)}"
         )
 
+
+class TestLagunaXS2(_LagunaModelTest):
+    """Keep the original Laguna XS.2 regression covered."""
+
+    NATIVE_MODEL_ID = os.environ.get("GPTQMODEL_LAGUNA_XS2_MODEL", "/monster/data/model/Laguna-XS.2")
+
     def test_laguna_xs2(self):
         self.quantize_and_evaluate()
+
+
+class TestLagunaS21(_LagunaModelTest):
+    """Exercise Laguna S 2.1's larger attention and singular shared-expert checkpoint layout."""
+
+    NATIVE_MODEL_ID = os.environ.get("GPTQMODEL_LAGUNA_S21_MODEL", "/monster/data/model/Laguna-S-2.1")
+    TRUST_REMOTE_CODE = True
+    # The first dense block and first sparse block span Laguna's two decoder
+    # topologies; avoid quantizing a redundant second sparse block on this 118B model.
+    MODEL_COMPAT_FAST_LAYER_COUNT = 2
+    MIN_FAST_MOE_LAYER_COUNT = 1
+    DATASET_SIZE_FAST = 16
+    # Keep routing aligned with the checkpoint instead of forcing all 256 experts
+    # active for every calibration token.
+    MOE_CONFIG = MoEConfig(routing=ExpertsRoutingOverride(num_experts_per_tok=10))
+    # The hybrid checkpoint retains untouched BF16 layers and needs a multi-GPU
+    # post-quant reload for Evalution.
+    EVAL_SINGLE_GPU = False
+    EVAL_TASKS_FAST = copy.deepcopy(_LagunaModelTest.EVAL_TASKS_FAST)
+    EVAL_TASKS_FAST["gsm8k_platinum_cot"]["evalution_suite_kwargs"]["max_rows"] = 8
+    EVAL_TASKS_FAST["gsm8k_platinum_cot"]["acc,num"]["value"] = 0.125
+    EVAL_TASKS_FAST["arc_challenge"]["evalution_suite_kwargs"]["max_rows"] = 8
+    EVAL_TASKS_FAST["arc_challenge"]["acc"]["value"] = 0.375
+    EVAL_TASKS_FAST["arc_challenge"]["acc_norm"]["value"] = 0.625
+
+    def test_laguna_s21(self):
+        self.quantize_and_evaluate()

tests/test_laguna_support.py

@@ -37,6 +37,9 @@ def test_laguna_module_tree_expands_dense_attention_and_sparse_moe_paths():
     assert "mlp.shared_experts.gate_proj" in flat_modules
     assert "mlp.shared_experts.up_proj" in flat_modules
     assert "mlp.shared_experts.down_proj" in flat_modules
+    assert "mlp.shared_expert.gate_proj" in flat_modules
+    assert "mlp.shared_expert.up_proj" in flat_modules
+    assert "mlp.shared_expert.down_proj" in flat_modules
     assert "mlp.experts.0.gate_proj" in flat_modules
     assert "mlp.experts.1.up_proj" in flat_modules
     assert "mlp.experts.2.down_proj" in flat_modules
@@ -55,9 +58,10 @@ def test_laguna_consumes_defuser_for_fused_expert_conversion():
         hidden_size=16,
         intermediate_size=32,
         num_attention_heads=2,
-        num_attention_heads_per_layer=[2, 2],
+        num_attention_heads_per_layer=[2, 4],
         num_key_value_heads=1,
         head_dim=8,
+        gating="per-head",
         vocab_size=32,
         num_experts=3,
         num_experts_per_tok=2,
@@ -67,6 +71,7 @@ def test_laguna_consumes_defuser_for_fused_expert_conversion():
         layer_types=["full_attention", "full_attention"],
     )
     model = modeling_laguna.LagunaForCausalLM(config)
+    assert model.model.layers[1].self_attn.g_proj.out_features == 4
     experts = model.model.layers[1].mlp.experts
     assert hasattr(experts, "gate_up_proj")
 

tests/test_qwen2_family_compat.py

@@ -17,7 +17,7 @@
 
 from gptqmodel.models.definitions import base_qwen2_5_omni, base_qwen2_vl
 from gptqmodel.utils import structure as structure_module
-from gptqmodel.utils.hf import load_tokenizer
+from gptqmodel.utils.hf import load_hf_tokenizer, load_tokenizer
 
 
 def test_qwen2_vl_image_only_process_vision_info_returns_image_list():
@@ -437,3 +437,35 @@ def get_text_config(self):
 
     assert any(item.category is DeprecationWarning for item in caught)
     assert wrapped.model_config is text_config
+
+
+def test_load_hf_tokenizer_delegates_normalization_to_tokenicer(monkeypatch):
+    native_tokenizer = object()
+    model_config = object()
+    calls = []
+
+    def fake_load(tokenizer_or_path, **kwargs):
+        calls.append((tokenizer_or_path, kwargs))
+        return types.SimpleNamespace(tokenizer=native_tokenizer)
+
+    monkeypatch.setattr(Tokenicer, "load", staticmethod(fake_load))
+
+    loaded = load_hf_tokenizer(
+        "/tmp/model",
+        model_config=model_config,
+        trust_remote_code=True,
+        revision="test-revision",
+    )
+
+    assert loaded is native_tokenizer
+    assert calls == [
+        (
+            "/tmp/model",
+            {
+                "model_config": model_config,
+                "trust_remote_code": True,
+                "revision": "test-revision",
+            },
+        )
+    ]
+    assert "fix_mistral_regex" not in calls[0][1]