Getting Chat-Instruct to reason on GPT-OSS

[yanghuaxuan]

On pure Instruct mode, reasoning with GPT-OSS using the in-model template from ggml-org/gpt-oss-20b-GGUF seems to "just work". It doesn't seem to just work on Chat-Instruct mode unfortunately.

One way to fix this is to simply "force" it to reason by appending <|channel|>analysis<|message|> at the end. So it would look something like:

<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2025-08-13

Reasoning: medium

# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>user<|message|>Continue the chat dialogue below. Write a single reply for the character "AI". 

The following is a conversation with an AI Large Language Model. The AI has been trained to answer questions, provide recommendations, and help with decision making. The AI follows user requests. The AI thinks outside the box.

AI: How can I help you today?
Sensei: Tell me a joke
<|end|><|start|>assistant<|channel|>analysis<|message|>

Generating this prompt outputs:

We need to continue the chat dialogue. The last line was "Sensei: Tell me a joke". The AI should reply with a joke. The instruction: "Write a single reply for the character 'AI'." So we need to produce the AI's reply. It should be a joke. The AI is trained to think outside the box. So maybe a clever joke. The context: the conversation is with an AI. The user is Sensei. The AI says "How can I help you today?" The user says "Tell me a joke". So AI replies with a joke. Let's produce a joke, maybe punny. Let's be creative. The AI might mention a pun about computers or something. Let's produce a short, witty joke. Something like: "Why don't programmers like nature? Too many bugs." But maybe something about sensei? Could be "What did the sensei say to the student? 'You're a little too sharp, you need to be more... dull?'" Eh.

We need a single reply. Let's produce a joke. We'll keep it short and witty. Let's go: "Why did the scarecrow become a successful AI? Because he was outstanding in his field." That might be okay. Or a pun about sensei: "Why did the sensei bring a ladder to class? Because the students were always looking for higher learning." Eh.

Let's produce a joke: "Why did the AI go to the beach? It wanted to surf the net." That's a good one. Let's produce that.<|start|>assistant<|channel|>final<|message|>AI: Why did the AI go to the beach? Because it heard the waves were full of great “surf‑the‑net” opportunities!

Now it reasons, and also correctly outputs in chat dialogue form!

This seems like the best solution for getting the model to reason on Chat-Instruct mode. But it seems neither Instruct or Chat-Instruct modes "force" the model to reason by shoving in <|channel|>analysis<|message|>. On Instruct mode, it just seems to like to reason by default.