Compatibility with other llm

[flefevre]

Is your feature request related to a problem? Please describe.
I and the open source community do not want to use private llm such as chatgpt

Describe the solution you'd like
My ideal is to be able to connect to llm model server such as https://github.com/ollama/ollama
Or https://github.com/vllm-project/vllm
Best through proxies https://github.com/BerriAI/litellm