Skip to content
This repository was archived by the owner on Jun 5, 2025. It is now read-only.

Fix FIM for OpenRouter #1097

Merged
merged 2 commits into from
Feb 19, 2025
Merged

Fix FIM for OpenRouter #1097

merged 2 commits into from
Feb 19, 2025

Conversation

aponcedeleonch
Copy link
Contributor

@aponcedeleonch aponcedeleonch commented Feb 19, 2025

FIM was not working with OpenRouter in Continue. The reason was that we get FIM requests in /completions. LiteLLM when using acompletion is forcing /chat/completions and the return format {..., "choices":[{"delta":{"content":"some text"}}]}. However, Continue was expecting the format: {..., "choices":[{"text":"some text"}]} becuase of the endpoint it called.

With this PR we force the return format to be the latter using atext_completion from LiteLLM

@aponcedeleonch aponcedeleonch changed the title Fix fim openrouter Fix FIM for OpenRouter Feb 19, 2025
jhrozek
jhrozek previously approved these changes Feb 19, 2025
Copy link
Contributor

@jhrozek jhrozek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, litellm. The gift that keeps on giving.

FIM was not working with OpenRouter in Continue. The reason
was that we get FIM requests in `/completions`. LiteLLM when
using `acompletion` is forcing `/chat/completions` and the return
format `{..., "choices":[{"delta":{"content":"some text"}}]}`.
However, Continue was expecting the format:
`{..., "choices":[{"text":"some text"}]}` becuase of the endpoint
it called.

With this PR we force the return format to be the latter using
`atext_completion` from LiteLLM
@aponcedeleonch aponcedeleonch merged commit d01f27c into main Feb 19, 2025
11 checks passed
@aponcedeleonch aponcedeleonch deleted the fix-fim-openrouter branch February 19, 2025 10:31
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants