Misc. bug: Server response "model" seems to be incorrect

[turtle0x1]

Name and Version

version: 4410 (4b0c638)
built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Problem description & steps to reproduce

The /completion end point use to emit a output.generation_settings.model with the correct value.

Its now output.model which is fine, but it seems to always say gpt-3.5-turbo which is not the model I'm using. I've also tried settings response_fields to see if that "tricks" it into giving the right value but it doesn't seem to.

When adding the model in the server tool it appears as the attached logs.

First Bad Commit

No response

Relevant log output

llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Hermes 3 Llama 3.1 8B
llama_model_loader: - kv   3:                       general.organization str              = NousResearch
llama_model_loader: - kv   4:                           general.basename str              = Hermes-3-Llama-3.1

[ngxson]

It's expected, and the old model field has been moved to /props

Ref: #10691

[turtle0x1]

It was nice having it per generation because requests might be submitted to a pool where it doesn't know which server its requests went to / came from and proxies didn't have to monitor /props, this change means they need to be aware and check periodically. Is returning the wrong name the only solution? Is it for eventual removal?

[ngxson]

It's remove because you can set model name via --alias argument. It doesn't make sense to use model path as model name (or if you really want, we can add is back as model_path field)

[turtle0x1]

I see, I think I never noticed it was a path because of the way I load the server.

Respectfully I think the right path was better than wrong value without setting a flag, but I see your point, model_path would probably be good but i'll leave that decision up to you. Now I know about --alias i'll probably program that into the workflow.

[ngxson]

If --alias works for you, then I think we don't need to add model_path anymore.

IMO --alias should be more suitable to your reverse proxy use case, since you can set it to hostname / container ID / etc.

[turtle0x1]

Great, thanks for the help!