Gemma4 moe pr v3#221

Draft

cavusmustafa wants to merge 135 commits into

ravi9:dev_backend_openvinofrom

cavusmustafa:gemma4_moe_pr_v3

Commits on Jun 9, 2026

Add interface is_model_splitted() to check the c-graph is splited or not

zhaixuejun1993
authored and
ravi9
committed
Infer and propagate dynamic-dimension indices for all tensors in the GGML graph in api compute_model_outputs()

zhaixuejun1993
authored and
ravi9
committed
Only do this for fallback sub graph

zhaixuejun1993
authored and
ravi9
committed
Move dynamic dims compute in graph missmatch

zhaixuejun1993
authored and
ravi9
committed
ggml-openvino: fix tensor data handling for PERMUTE/VIEW ops in split models

zhaixuejun1993
authored and
ravi9
committed
ggml-openvino:add comments

zhaixuejun1993
authored and
ravi9
committed
ggml-openvino: override VIEW op_case to 0 for split model inputs

zhaixuejun1993
authored and
ravi9
committed
openvino backend: Handle unsupported VIEW shape-mismatch in OpenVINO backend

zhaixuejun1993
authored and
ravi9
committed
Enable additional mul_mat tests and add tensor data saving function (#81)

zhaixuejun1993
authored and
ravi9
committed
ggml-openvino: fix CONT/TRANSPOSE mapping and improve dynamic-dimension handling

zhaixuejun1993
authored and
ravi9
committed
OpenVINO: add NORM/TANH support and rework SOFT_MAX translation

zhaixuejun1993
authored and
ravi9
committed
ggml-openvino: extend VIEW handling

zhaixuejun1993
authored and
ravi9
committed
Enable -fa off (#118)

wine99
authored and
ravi9
committed
Enable --context-shift

wine99
authored and
ravi9
committed
Fix llm param compute error for normal softmax not the softmax in attention

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: fix error for attention size compute in llm param

zhaixuejun1993
authored and
ravi9
committed
use tensor->extra in infer_request i/o

wine99
authored and
ravi9
committed
OpenVINO backend: refacter the compute_llm_params() func add get_attention_pattern_case to easy extand

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: clean unused code

zhaixuejun1993
authored and
ravi9
committed
1to1 match op update (#146)

authored and
ravi9
committed
initial gemma4 support

Mustafa Cavus
authored and
ravi9
committed
removed hardcoded names for kv cache slicing

cavusmustafa
authored and
ravi9
committed
OpenVINO backend: Add new attention pattern for llm parameters compute

zhaixuejun1993
authored and
ravi9
committed
flash attn Q shape static conversion

cavusmustafa
authored and
ravi9
committed
Remove slice in permute translation when n_seq is 1

cavusmustafa
authored and
ravi9
committed
return optional in extract_layer_from_name

wine99
authored and
ravi9
committed
OpenVINO backend: refactor VIEW related operation (#148)

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: Add ops l2_norm & pad

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend does not support CPY with non-contiguous data or mismatched types

zhaixuejun1993
authored and
ravi9
committed
add op SSM_CONV GATED_DELTA_NET

wine99
authored and
ravi9
committed
OpenVINO backend: fix error for bf16 in OV gpu plugin

zhaixuejun1993
authored and
ravi9
committed
reverted static Q input shape for attention layer

cavusmustafa
authored and
ravi9
committed
OpenVINO backend: remove hardcode name inp_tokens, which ignore some leaf case

zhaixuejun1993
authored and
ravi9
committed
Disable remote tensor due to bug in ov gpu

wine99
authored and
ravi9
committed
Disable n_token > 1 GATED_DELTA_NET on gpu

wine99
authored and
ravi9
committed
OpenVINO backend: fix the view op dynamic handling issue in gemma4 & enable view + get_row

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: clean code

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable view + norm/rms_norm

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: concat op

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: argsort op

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable unary + view & GGML_UNARY_OP_SOFTPLUS

zhaixuejun1993
authored and
ravi9
committed
Fix issue for test-backend-ops in TOPK_MOE, which compare VIEW ops result, VIEW node in OpenVINO no need compare, the whole graph result is correct

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable sum_rows

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable clamp

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable DIV

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable GGML_OP_MUL_MAT_ID

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: disable MUL_MAT_ID_FUSION case with large mem needed

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: Disable GGML_OP_ARGSORT, cause test_backend-ops failed

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: fix issue in mul_mat_id

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: Disable DIV with broadcast on GPU

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: update DIV

zhaixuejun1993
authored and
ravi9
committed
use ov internal op GatedDeltaNet

wine99
authored and
ravi9
committed
OpenVINO backend: enable llama erch test qwen3next

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable RMS_NORM + VIEW & remove op_case 2 for rope

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: fix error

zhaixuejun1993
authored and
ravi9
committed
suggested changes, need review

wine99
authored and
ravi9
committed
suggested changes, need review

wine99
authored and
ravi9
committed
OpenVINO backend: clean unused code & fix build warning

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable minicpm3 for arch test

zhaixuejun1993
authored and
ravi9
committed
Disable GDN op (#177)

wine99
authored and
ravi9
committed
disable gated_delta_net

wine99
authored and
ravi9
committed
update stateful_kv_size correctly in mismatch case

wine99
authored and
ravi9
committed
OpenVINO backend: enable arch test for qwen3vl

Xuejun
authored and
ravi9
committed
OpenVINO backend: enable cohere2 for arch test

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable t5 for arch test

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable jamba for arch test

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: remove warning for tmp

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable kimi-linear for arch test

zhaixuejun1993
authored and
ravi9
committed
Remove unused

zhaixuejun1993
authored and
ravi9
committed
Fix gpt-oss accuracy issue

yangwang201911
authored and
ravi9
committed
OpenVINO backend: enable arctic for arch test

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: enable grok for arch test

zhaixuejun1993
authored and
ravi9
committed
Gemma4 initial npu support (#179)

cavusmustafa
authored and
ravi9
committed
ggml-openvino: add GGML_OPENVINO_ENABLE_CACHE env var to control decoder cache. Add environment variable GGML_OPENVINO_ENABLE_CACHE (default: YES). When set to NO, the decoder_cache is bypassed and…

zhaixuejun1993
authored and
ravi9
committed
Revert "Gemma4 initial npu support (#179)"

wine99
authored and
ravi9
committed
OpenVINO backend: disable debug log print

zhaixuejun1993
authored and
ravi9
committed
Update TBB discovery. Delegated to OpenVINOs own config.
ravi9
committed
OpenVINO backend: GGML_OPENVINO_ENABLE_CACHE YES -> 1

zhaixuejun1993
authored and
ravi9
committed
OpenVINO backend: fallback FLASH_ATTN_EXT in gemma3n to CPU backend

zhaixuejun1993
authored and
ravi9
committed
Add raw ov infer profiling metric

virajwad
authored and
ravi9
committed
Add OV raw infer time metric to static compute path

authored and
ravi9
committed
Modify precision of static profiling

virajwad
authored and
ravi9
committed
update to OV 2026.2, add OV windows CI
ravi9
committed
fix editorconfig-checks
ravi9
committed
Initiall gemma4 npu support

cavusmustafa
authored and
ravi9
committed
temp. fix for gemma4 accuracy bug on npu

cavusmustafa
authored and
ravi9
committed
Remove hardcoded names for npu-fold handling

cavusmustafa
authored and
ravi9
committed
revert static n tokens for cont translation as it is not needed

cavusmustafa
authored and
ravi9
committed
removed unused variable

cavusmustafa
authored and
ravi9
committed
test-llama-archs fix

cavusmustafa
authored and
ravi9
committed
Fix gemma4 flash_attn fallback

cavusmustafa
authored and
ravi9
committed
support im2col

mostafafaheem
authored and
ravi9
committed
fix code style

mostafafaheem
authored and
ravi9
committed
disable add_rope_sin_cos optimization

wine99
authored and
ravi9
committed
stateless boradcast and rope optimizations

cavusmustafa
authored and
ravi9
committed
Enable manual gqa attn by default for stateless gpu

cavusmustafa
authored and
ravi9
committed
manual gqa: fixed static batch

Mustafa Cavus
authored and
ravi9
committed
gemma4 llama-bench ctx update fix

cavusmustafa
authored and
ravi9
committed
Update OV win CI
ravi9
committed
stateful rope fusion temp. fix

cavusmustafa
authored and
ravi9
committed
OpenVINO backend: Conslolidate supported ops

mostafafaheem
authored and
ravi9
committed
Exclude unsupported GGML_OP_SUB cases

mostafafaheem
authored and
ravi9
committed
Exclude unsupported TOPK_MOE cases

mostafafaheem
authored and
ravi9
committed
OpenVINO Backend: MUL_MAT enhancements

mostafafaheem
authored and
ravi9
committed
Update OV CI
ravi9
committed
support f16 mask input for npu

wine99
authored and
ravi9
committed
Make GGML_OPENVINO_* env vars usage uniform
ravi9
committed
OpenVINO backend: Enhance envvar handling
mostafafaheem
committed
more cleanup
mostafafaheem
committed
move ggml_openvino_env_flag to appropriate place
mostafafaheem
committed
OpenVINO backend: add REPEAT translator, Q5_1 weights, and GLU view-input fix
cavusmustafa
committed
Merge pull request #208 from mostafafaheem/envvar_cleanup
ravi9
authored
ggml-openvino: fix -Werror=cast-qual in extract_q5_1_data
cavusmustafa
committed
Merge pull request #209 from cavusmustafa/op_translations_q51
ravi9
authored

Commits on Jun 13, 2026

Merge branch 'master' into dev_backend_openvino
ravi9
authored

Commits on Jun 15, 2026

Commits on Jun 16, 2026

Commits on Jun 18, 2026

ggml-openvino: fix op-test regressions (MUL_MAT_ID large-tmp cap + q4_1/q5_1 GET_ROWS)
cavusmustafa
committed

Commits

Commits on Jun 9, 2026

Commits on Jun 10, 2026

Commits on Jun 12, 2026

Commits on Jun 13, 2026

Commits on Jun 15, 2026

Commits on Jun 16, 2026

Commits on Jun 17, 2026

Commits on Jun 18, 2026