llama: R-SWA reference sliding window attention for Unlimited-OCR

[sfallah]

Overview

Vibe-coded DRAFT to open the discussion about R-SWA support in llama.cpp.

Stacked on #24969 (the Unlimited-OCR converter) - that needs to be reviewed/merged first.

• Adds baidu/Unlimited-OCR to mtmd: DeepSeek-OCR v1 with the original R-SWA decoder.
• R-SWA: each token attends the full prefix (image + prompt) + a window over the last n_swa generated tokens; unlike plain SWA, the image stays visible.
• New llama_swa_type REFERENCE; L_m latched implicitly in the KV cache at the prefill->decode boundary.
• Single full KV cache (no eviction); n_ref maintained across the cache lifecycle ops.

Scope: single-page parity.

Validation

The correctness is validated by reusing/extending a regression test that was introduced for DeepSeek-OCR (v1+2) against the HF reference implementation.

Single-page, bf16, against my HF reference scoring (transformers 4.46.3):

	CER
llama.cpp	0.1397
HF reference	0.1869

Design note

R-SWA lives in core: LLAMA_SWA_TYPE_REFERENCE + the mask rule + per-seq n_ref in llama_kv_cache, activated only for deepseek2-ocr. Alternative: implement in deepseek2-ocr model code. Open to moving it into the model code if it is preferred.
Anyhow my immediate aim is that Unlimited-OCR is supported in llama.cpp with its original R-SWA.

How to run

GGUF models: sabafallah/Unlimited-OCR-GGUF

build/bin/llama-mtmd-cli -hf sabafallah/Unlimited-OCR-GGUF:bf16 \
  --image tools/mtmd/test-1.jpeg -p "document parsing." \
  --chat-template deepseek-ocr \
  --temp 0 --flash-attn off --no-warmup \
  -n 4096 -c 16384 \
  --dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 \
  --dry-penalty-last-n -1 --dry-sequence-breaker none

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - this is largely an AI-written first draft, opened to start the discussion. I designed the R-SWA approach (from the model's paper), directed and reviewed the implementation; the code itself is mostly AI-generated. I take responsibility for the contents.

[jason-ni]

It seems the impl in this branch has numeric precision issue:
https://huggingface.co/sabafallah/Unlimited-OCR-GGUF/discussions/1

@o7si could you please have a test on image in that discussion? Thanks.

[sfallah]

It seems the impl in this branch has numeric precision issue: https://huggingface.co/sabafallah/Unlimited-OCR-GGUF/discussions/1

@o7si could you please have a test on image in that discussion? Thanks.

@jason-ni
yes, I will.
Thanks for reporting this.