How to Configure a Custom LLM Proxy in OpenOPC (and Debug LiteLLM)

(If you missed it, check out my previous post on Getting Started with OpenOPC to see why I’m using this agent framework in the first place!)

I recently started setting up OpenOPC to run my AI agents. To save on API costs, I decided to hook it up to a custom, OpenAI-compatible proxy at https://api.freetheai.xyz/v1.

I expected it to be a simple drop-in replacement. In other platforms, you just set your model to something like glm/glm-5.2 and it works perfectly. So, I updated my .opc/config/llm_config.yaml:

llm:
  default_model: "glm/glm-5.2"
  api_base: "https://api.freetheai.xyz/v1"
  api_key: "my_api_key"

Enter fullscreen mode Exit fullscreen mode

OpenOPC uses LiteLLM under the hood to handle model routing. Instead of my agents spinning up, LiteLLM crashed out with this error:

litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=glm/glm-5.2

The openai/ Prefix Trick

Because I was pointing to a custom api_base, LiteLLM didn’t natively know how to route the glm/ provider prefix. It needs to know exactly which API format to use to talk to the endpoint.

For almost all custom endpoints, the format is OpenAI-compatible. To force LiteLLM to use the standard OpenAI request format, you have to prefix your model with openai/.

I updated my config:

llm:
  default_model: "openai/glm-5.2"

Enter fullscreen mode Exit fullscreen mode

The “Unknown Aliased Model” Trap

I restarted OpenOPC. LiteLLM knew how to format the request this time, but I hit a new error. This time, it was rejected by the proxy itself:

OpenAIException - unknown aliased model

When you use the openai/ prefix, LiteLLM strips it off before sending the request. That means it passed exactly glm-5.2 to the proxy. The proxy had no idea what that was.

To see what the proxy actually expected, I queried its models endpoint directly using curl:

curl -s https://api.freetheai.xyz/v1/models -H "Authorization: Bearer my_api_key"

Enter fullscreen mode Exit fullscreen mode

Buried in the massive JSON response, I found the literal ID registered on the server for that model: glm/glm-5.2.

The Final Working Config

To make this work, I needed LiteLLM to use the OpenAI API format, but I needed it to pass glm/glm-5.2 as the actual model name in the JSON payload. The fix was stacking them:

llm:
  default_model: "openai/glm/glm-5.2"

Enter fullscreen mode Exit fullscreen mode

Once I restarted OpenOPC with this stacked config, the routing worked perfectly. I did hit a ServiceUnavailableError - provider capacity temporarily unavailable immediately after, but that’s just the reality of using busy proxy servers—the actual connection and routing were finally correct.

If you’re hooking OpenOPC up to a custom proxy, always hit their /v1/models endpoint first to get the literal string they expect, and slap openai/ on the front.

Discussion: How are you handling custom API proxies and model routing for your agent stacks? Do you prefer configuring prefixes like this in LiteLLM, or do you rewrite the requests at a proxy level before they even hit your models? Let me know below!

Looking for a solid API proxy? You can join FreeTheAI.xyz here.