Fix dead image branch in OpenAICompatible._conversation_to_list

[chuenchen309]

The image-modality branch in OpenAICompatible._conversation_to_list is unreachable:

if "image" in turn.content.data_type:   # data_type is a tuple!

data_type is a (mimetype, encoding) tuple from mimetypes.guess_type (see attempt.py), e.g. ("image/gif", None). Membership on a tuple checks element equality, so "image" in ("image/gif", None) is always False. The image branch is therefore never taken — every image turn falls through to the audio elif (whose pattern fails on the mimetype subtype) and then to the else, which raises:

garak.exception.GarakException: Data type image/gif not supported.

So all image/multimodal probes (e.g. visual_jailbreak) crash against OpenAI-compatible vision generators (gpt-4o, and any OpenAICompatible subclass such as NIM/Groq). The audio branch two lines below already does this correctly with data_type[0].

Fix

• Test the mimetype string: if "image" in turn.content.data_type[0]: (mirroring the audio branch).
• Add the missing comma in the base64 data URI — data:{mime};base64,{data} per RFC 2397. The now-reachable branch previously built ;base64{data} (no comma), which is a malformed data URI.

Scope kept minimal — the "input_text"/"input_image" payload keys are left as-is.

Verification

• Added test_conversation_to_list_image_turn_is_reachable: an image turn (using the existing tests/_assets/tinytrans.gif) must produce an input_image content part with a well-formed data:image/gif;base64,... URI instead of raising GarakException. Fails before the change (raises GarakException), passes after. Full test_openai_compatible.py passes; black --check clean.
• garak -t <target_type> -n <model_name> — not run: this is a unit-level fix and I have no live target configured; verified via the tests above rather than ticking a box I didn't exercise.
• Supporting configuration such as a generator configuration file — n/a, no config surface changed.

---

Disclosure: this contribution was prepared with the assistance of an AI agent (Claude Code). The analysis, fix, and test were reviewed by me before submission.

[jmartin-tech]

There is definitely a confirmed bug here, the initial code here conflated the chat/completions API and responses API and is in a weird limbo. #1806 targets at adding support for the responses API. For this PR it would be good to focus on support for chat completions and specifcially on getting the format right for image_url type inputs

Testing suggests this is incomplete, the text input need to be "type: "text" and the image needs to be "type": "image_url" with a url entry that contains the base64 data per the error code returned by OpenAI:

Error code: 400 - {'error': {'message': "Invalid value: 'input_text'. Supported values are: 'text', 'image_url', 'input_audio', 'refusal', 'audio', and 'file'.", 'type': 'invalid_request_error', 'param': 'messages[0].content[0].type', 'code': 'invalid_value'}}

Error code: 400 - {'error': {'message': "Invalid value: 'input_image'. Supported values are: 'text', 'image_url', 'input_audio', 'refusal', 'audio', and 'file'.", 'type': 'invalid_request_error', 'param': 'messages[0].content[1].type', 'code': 'invalid_value'}}

Note even after making these adjustments consumer OpenAI endpoints are returning an error when testing with the visual_jailbreak probe:

{'error': {'message': 'Invalid content type. image_url is only supported by certain models.', 'type': 'invalid_request_error', 'param': 'messages.[0].content.[1].type', 'code': None}}

If there is a model that can be hosted via VLLM and fully supports this chat API format that may be the path to getting this validated as working.

[chuenchen309]

Thanks — you're right, and thank you for testing it against a live endpoint. I kept the original input_text/input_image spelling when fixing the tuple check, without questioning whether it was the correct one for this call path; your 400s show it is the responses API shape.

Pushed: the image branch now emits the chat/completions shape — {"type": "text"} plus {"type": "image_url", "image_url": {"url": "data:<mimetype>;base64,<data>"}} with image_url as an object rather than a bare string. The adjacent audio branch already emits {"type": "text"}, so the two are now consistent. Test updated to assert the part types and the image_url.url shape. Responses API support left to #1806.

On your last point — I have no way to verify against a live endpoint or a VLLM-hosted vision model from here, so I can't confirm the image_url is only supported by certain models behaviour you saw. My testing is limited to the unit test asserting the emitted payload shape against the chat/completions schema. Happy to adjust further if the live test still shows a mismatch.

[chuenchen309]

Pushed — the branch now uses the chat/completions spellings you gave (text / image_url, with image_url as an object carrying url), and touches nothing on the responses-API side, so it stays out of #1806's way.

Your 400s also surfaced a second bug in that branch: the data URI was built as data:image/gif;base64<payload> — no comma — so even with the type names corrected the payload would have been malformed. The test now base64-decodes the url and asserts it round-trips to the original asset bytes, which pins both the separator and the encoding.

For the nesting I checked the pinned SDK rather than going from memory — openai/types/chat/chat_completion_content_part_image_param.py declares image_url: Required[ImageURL] with url: Required[str], so it's an object, not a bare string.

One thing I can't close out: I have no access to a model that accepts image_url on chat/completions, so I can't confirm the end-to-end path you hit with visual_jailbreak. Everything here is verified against the SDK's own types and the unit tests (tests/generators/: 341 passed, 17 skipped) — not against a live endpoint. If the VLLM route is the way to validate that, I'm happy to leave this open until someone with that setup can confirm.

Also flagging one thing I deliberately left alone: an unknown mimetype (guess_type returning (None, None)) still crashes rather than raising GarakException — pre-existing, but the exception type shifts from AttributeError to TypeError with this change. Seemed like scope creep against "focus on chat completions"; happy to fix it here or in a separate PR, whichever you prefer.