Should ModelPack standardize a Modelfile format?

[ericcurtin]

Background

A contributor recently opened docker/model-runner#903 to add Modelfile support to Docker Model Runner. The implementation follows Ollama's Modelfile format, which is oriented around runtime configuration (inference parameters, prompt templates, system prompts, etc.).

At the same time, modctl ships its own Modelfile format (via modctl modelfile generate), which serves a different purpose: describing packaging metadata for OCI artifacts (architecture, format, precision, weights, configs, etc.).

This gives us two things both called "Modelfile":

	modctl Modelfile	Ollama Modelfile
Purpose	OCI packaging metadata	Runtime/inference configuration
Key fields	NAME, ARCH, FORMAT, MODEL, CONFIG	FROM, PARAMETER, TEMPLATE, SYSTEM

Questions for the community

Does ModelPack plan to formally specify a Modelfile format as part of this spec?

[bergwolf]

The modelpack spec is to describe a model packaged in OCI format. Modelfile in modctl's case is to describe how to build the artifacts, while Modelfile in Ollama's case is for runtime to consume it. Right now they are actually two different files serving different purpose. Are you concerning the filename conflict or do you see some issue with the two Modelfiles?

On specifying a Modelfile format as part of this spec, IMO we should be open to it. Right now, there are multiple packaging tools (modctl/AIKit/KitOps). If there are interests in defining a common Modelfile format, we should go for it.

[ericcurtin]

The modelpack spec is to describe a model packaged in OCI format. Modelfile in modctl's case is to describe how to build the artifacts, while Modelfile in Ollama's case is for runtime to consume it. Right now they are actually two different files serving different purpose. Are you concerning the filename conflict or do you see some issue with the two Modelfiles?

I'm not concerned about the conflict.

On specifying a Modelfile format as part of this spec, IMO we should be open to it. Right now, there are multiple packaging tools (modctl/AIKit/KitOps). If there are interests in defining a common Modelfile format, we should go for it.

This is what I'm interested in, a CNCF/ModelPack defined standard. Ollama do lots of non-standard things, non-standard OCI artifacts, non-standard OCI transport, non-standard gguf files. So I'm interested in a CNCF/ModelPack defined standard of Modelfile.

[gorkem]

@ericcurtin The naming collision is unfortunate but worth separating from the question of scope.
Kitfiles in KitOps are packaging manifests, not runtime configs. The same is true for modctl's Modelfile. Ollama's Modelfile is a different artifact entirely: it describes how to run a model, not how to package one.

ModelPack is the packaging spec. It is not going to define a runtime config format and it should not be called Modelfile if it ever did.

What ModelPack can do, and arguably should, is extend the artifact's own metadata to carry runtime hints (chat template, recommended context length, stop tokens, etc.) as part of the config descriptor or annotations. Runtimes that consume ModelPack artifacts can then configure themselves directly from the artifact, without a separate Modelfile of either flavor. That is a different proposition from standardizing a Modelfile syntax.

Is the runtime-properties direction what you are looking for, or were you asking about a packaging Modelfile?

[ericcurtin]

@ericcurtin The naming collision is unfortunate but worth separating from the question of scope. Kitfiles in KitOps are packaging manifests, not runtime configs. The same is true for modctl's Modelfile. Ollama's Modelfile is a different artifact entirely: it describes how to run a model, not how to package one.

ModelPack is the packaging spec. It is not going to define a runtime config format and it should not be called Modelfile if it ever did.

Ok lets say this variant is called Llmfile for the sake of it (I'm just trying to avoid the naming discussion, but I'm genuinely happy with any old name).

What ModelPack can do, and arguably should, is extend the artifact's own metadata to carry runtime hints (chat template, recommended context length, stop tokens, etc.) as part of the config descriptor or annotations. Runtimes that consume ModelPack artifacts can then configure themselves directly from the artifact, without a separate Modelfile of either flavor. That is a different proposition from standardizing a Modelfile syntax.

Is the runtime-properties direction what you are looking for, or were you asking about a packaging Modelfile?

I'm open minded, in DMR we have "docker model package various args". But I think a standardized "Llmfile" would be worth it. And if anything if gives some level familiarity with ollama, which users crave. Now ours can be completely different to Ollama's Modelfile in the content required, but I think a standard is worth it

[ericcurtin]

# Llmfile — runtime inference configuration

# Required: OCI model reference (ModelPack artifact or registry tag)
FROM smollm2:latest

# Optional: operational mode (default: completion)
MODE completion  # completion | embedding | reranking | image-generation

# Optional: system prompt
SYSTEM You are a helpful, concise assistant.

# Optional: Jinja2 chat template (overrides model default)
TEMPLATE """
{{- if .System }}<|system|>
{{ .System }}<|end|>
{{- end }}
<|user|>
{{ .Prompt }}<|end|>
<|assistant|>
"""

# Optional: runtime parameters
PARAMETER context_size 8192
PARAMETER keep_alive 5m
PARAMETER think false

# Speculative decoding
PARAMETER speculative_draft_model ai/smollm2-draft:latest
PARAMETER speculative_num_tokens 5
PARAMETER speculative_min_acceptance_rate 0.9

[ericcurtin]

@areebahmeddd @ilopezluna @doringeman thoughts?

[ilopezluna]

I like the idea of standardizing a ModelPack "Llmfile".

My main question is whether this file is intended to be a one-to-one representation of a ModelPack OCI artifact:

• From an Llmfile, can I deterministically create one OCI artifact?
• From an OCI artifact, can I recover the equivalent Llmfile?
• If yes, does the Llmfile become another representation of the model artifact itself?

I think this could make adoption of ModelPack easier. Consumers would not need to construct OCI artifacts using random parameters, ad hoc conventions, or tool-specific mechanisms. Instead, they could consume a well-defined file format and map it to an OCI artifact. That would also make it easier to build SDKs around the standard.

Another benefit is that files are easy to version, review, diff, and store in source control, which could help make model packaging workflows more reproducible.

[doringeman]

Hello everyone,

+1 on a standard runtime config file. I like the idea of having some parameters, such as chat template, stop tokens, context part of the artifact, and the file layers runtime overrides on top (system prompt, speculative decoding, etc.).
I see the file (Llmfile or whatever) as what you'd use in Compose, as how you run the model; the artifact is the model.

[areebahmeddd]

Hi all, sharing my 2 bits here..

to add on what gorkem and doringeman mentioned, i think runtime hints that are model-intrinsic like chat templates stop tokens and context length should live in the oci artifact itself. then a separate config file can handle deployment-level overrides on top. it would make artifacts more portable and self-describing 🙂

one open question is that there still isn’t a standard way to take local files like gguf safetensors chat templates etc. and turn them into an oci artifact

[ericcurtin]

Hi all, sharing my 2 bits here..

to add on what gorkem and doringeman mentioned, i think runtime hints that are model-intrinsic like chat templates stop tokens and context length should live in the oci artifact itself. then a separate config file can handle deployment-level overrides on top. it would make artifacts more portable and self-describing 🙂

one open question is that there still isn’t a standard way to take local files like gguf safetensors chat templates etc. and turn them into an oci artifact

This is the CNCF ModelPack group that defines the OCI artifact standard, that's already defined (can be further evolved of course). Now we are talking about how Llmfile would translate to this.

[bergwolf]

Is the runtime-properties direction what you are looking for, or were you asking about a packaging Modelfile?

@ericcurtin +1, I am definitely interested in adding runtime-properties. It would be nice to have some built-in properties in the packaged model image, and then have another config on top of the packaged ones for deployment tunables.

[aftersnow]

I think it helps to separate this into three layers.

First, there is model artifact configuration. This is what ModelPack already covers: what the model is, what format it uses, how weights/config/docs/code/datasets are described as OCI layers, and how metadata like license, precision, quantization, and capabilities are captured in the schema. This should be stable, verifiable, and readable by registries, runtimes, and platforms.

Second, there is the package/build recipe. This is about how to create a ModelPack artifact from sources like Hugging Face models, local GGUF/Safetensors files, or existing artifacts. modctl modelfile generate feels closer to this layer, and Docker’s docker model package --gguf/--safetensors-dir/--from also seems to fit here. This is more like a Dockerfile or a build manifest.

Third, there is the runtime/serving profile. This is about how the model should run by default when served: context size, temperature, top_p, stop sequences, chat template, system prompt, adapters, tool-calling support, structured output, preferred engine, resource hints, etc. Ollama’s Modelfile mostly sits in this layer, although it also includes build/derivation concepts like FROM and ADAPTER.

So I think the real question is: should and how ModelPack standardize this third layer as well?

My answer would be yes, but as a separate runtime spec, not just as “the Modelfile format”.

[gorkem]

I think we should explore standardizing a runtime/inference profile: a typed, declarative description of how a model is intended to be served, kept separate from the packaging metadata in the config object. A profile is not a property of the weights. It is mutable, there can be several per model (a deterministic profile, a long-context profile), and it should never force a new model config digest. That rules out putting it in config.v1+json.

Concretely, three things need to be defined, and they are not independent.

1. How the profile is classified in the artifact. ModelPack expresses content kind through the layer media type (weight, doc, code, dataset), with annotations carrying file-level metadata, not semantic kind. Do we need to define a way to classify the profile artifact?

2. This should be a small behavioral core, not a parameter encyclopedia. The core holds the handful of sampling fields that every serving runtime genuinely implements, each tagged with a conformance level (MUST honor, SHOULD honor, MAY ignore) and marked behavioral or hint. The chat template and modality preprocessing config should be referenced by digest, not re-encoded, since the de facto standards (generation_config.json, the Jinja chat template, preprocessor_config.json) already exist and are read by vLLM, TGI, and transformers today. Anything the core cannot express goes in an explicitly engine-scoped block that only the matching runtime reads. ModelPack owns the core. It does not own the engine-specific dialects, and it must not duplicate fields already in the model config.

3. The mapping from the profile to each runtime. This is the part that determines whether any of this is worth doing.
   ModelPack has to own the resolvers, not just the schema. A profile format with no mapping is not a standard, it is a suggestion, and runtimes will diverge silently. The reasons are concrete. The same parameter has different names across engines (num_ctx vs max_model_len vs n_ctx). Worse, the same name is sometimes a different algorithm. llama.cpp's repeat_penalty is not OpenAI's presence_penalty/frequency_penalty, and sampler ordering changes the output distribution for identical setting. These are not fixable with a rename table, which is exactly why the mapping has to be owned and tested rather than left to each runtime. If we publish a structure and leave the interpretation to each runtime, the same profile produces different behavior depending on where it runs, which defeats the reason for standardizing it.

   So ModelPack should ship, as part of the spec, at minimum a reference resolver and a conformance test suite with input profiles and expected behavior. Runtimes may implement their own resolvers, but they pass conformance or they are not conformant.

[github-actions]

This issue is stale because it has been open 40 days with no activity.