A bare 'custom' provider with a remote base_url is a vendor-routing proxy (LiteLLM, Bedrock gateway, etc.). A slashed model id like bedrock/opus-4-6 is intrinsic — the proxy routes on the full string. The prior heuristic stripped ANY known-provider prefix (bedrock is also a provider name), truncating it to opus-4-6 and causing 403 'model not allowed'. Fix: in the custom-base_url branch, strip the prefix only when (a) the configured provider is a real first-party provider pointed at an OpenAI-compatible proxy (provider=openai + proxy base_url — the #433 path), OR (b) for a bare 'custom' provider, the bare id is genuinely a first-party model of that prefix's catalog (openai/gpt-5.4 -> gpt-5.4, since gpt-5.4 IS an OpenAI model). An intrinsic routing prefix whose bare id is NOT first-party of that namespace is preserved (bedrock/opus-4-6 stays whole). Unknown prefixes (zai-org/...) preserved as before. Both prior behaviors work in tandem with the fix: - #433 sprint40: bare custom + openai/gpt-5.4 -> gpt-5.4 (redundant, strip) - #433 model_resolver: provider=openai + google/gemma -> gemma (proxy strip) - #548: custom + zai-org/GLM-5.1 -> preserved (unknown prefix) - #1625: custom + loopback base_url -> preserved (local server) - #3872: custom + bedrock/opus-4-6 -> preserved (intrinsic vendor prefix) Adds regression tests covering all of the above. Co-authored-by: nesquena-hermes <nesquena-hermes@users.noreply.github.com> Co-authored-by: haolf000 <haolf000@users.noreply.github.com>
This commit is contained in:
@@ -3,6 +3,10 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Custom providers with a remote `base_url` no longer truncate model IDs that contain a `/`.** A `custom` provider (bare or named `custom:<slug>`) pointed at a vendor-routing proxy (LiteLLM, a Bedrock gateway, etc.) is now passed the model ID exactly as selected when the prefix is an intrinsic routing segment — e.g. `bedrock/opus-4-6` reaches the proxy whole instead of being silently truncated to `opus-4-6`, which caused `403 model "opus-4-6" is not allowed`. A prefix that is genuinely redundant with the model's own first-party namespace is still stripped (`openai/gpt-5.4` → `gpt-5.4`), and the strip for real first-party providers pointed at an OpenAI-compatible proxy (e.g. `provider: openai`) is unchanged. If your proxy registers a model under a first-party-prefixed key (e.g. `anthropic/claude-opus-4.6`), list it under a `custom_providers[]` entry to route the full id through. (#3872)
|
||||
|
||||
## [v0.51.348] — 2026-06-10 — Release LL (Phase 0 hotfix: timeout regression + data-loss + leaks)
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -1695,6 +1695,30 @@ def _is_local_server_provider(provider_id: str) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _is_first_party_model(provider_id: str, model_id: str) -> bool:
|
||||
"""True when ``model_id`` is listed in ``provider_id``'s own static catalog.
|
||||
|
||||
Used to tell a *redundant* first-party prefix from an *intrinsic* routing
|
||||
prefix on a bare ``custom`` endpoint. ``openai/gpt-5.4`` → gpt-5.4 is a real
|
||||
OpenAI model, so ``openai/`` is a redundant leftover and strippable (#433).
|
||||
But ``bedrock/opus-4-6`` → opus-4-6 is NOT in bedrock's first-party catalog
|
||||
(those ids look like ``global.anthropic.claude-…``), so ``bedrock/`` is a
|
||||
vendor-routing segment a proxy needs whole (#3872). Returns False on any
|
||||
unknown provider or empty model so callers preserve the id.
|
||||
"""
|
||||
provider = str(provider_id or "").strip().lower()
|
||||
model = str(model_id or "").strip()
|
||||
if not provider or not model:
|
||||
return False
|
||||
catalog = _PROVIDER_MODELS.get(provider)
|
||||
if not isinstance(catalog, list):
|
||||
return False
|
||||
return any(
|
||||
isinstance(entry, dict) and entry.get("id") == model
|
||||
for entry in catalog
|
||||
)
|
||||
|
||||
|
||||
def _base_url_points_at_local_server(base_url: str) -> bool:
|
||||
"""True if base_url's host is a loopback or private IP (likely local server).
|
||||
|
||||
@@ -1998,13 +2022,37 @@ def resolve_model_provider(model_id: str) -> tuple:
|
||||
if (_is_local_server_provider(config_provider)
|
||||
or _base_url_points_at_local_server(config_base_url)):
|
||||
return model_id, config_provider, config_base_url
|
||||
# Only strip the provider prefix when it's a known provider namespace
|
||||
# (e.g. "openai/gpt-5.4" → "gpt-5.4" for a custom OpenAI-compatible proxy).
|
||||
# Unknown prefixes (e.g. "zai-org/GLM-5.1" on DeepInfra) are intrinsic to
|
||||
# the model ID and must be preserved — stripping them causes model_not_found.
|
||||
if prefix in _PROVIDER_MODELS:
|
||||
# Strip the provider prefix only when it's a known provider namespace
|
||||
# AND stripping is the right call for this configured provider:
|
||||
#
|
||||
# * A real first-party provider pointed at an OpenAI-compatible proxy
|
||||
# (e.g. provider=openai + base_url=litellm) expects the bare id —
|
||||
# "openai/gpt-5.4" → "gpt-5.4", "google/gemma-…" → "gemma-…". This
|
||||
# is the #433 behaviour and applies whenever config_provider is not
|
||||
# the bare "custom" pseudo-provider.
|
||||
#
|
||||
# * A *bare* ``custom`` provider (or a named ``custom:<slug>``) is a
|
||||
# vendor-routing proxy (LiteLLM, Bedrock gateway, OpenRouter-style
|
||||
# multi-vendor endpoint). There we strip ONLY a prefix that is
|
||||
# redundant with the model's own first-party namespace
|
||||
# ("openai/gpt-5.4" → gpt-5.4, since gpt-5.4 is genuinely an OpenAI
|
||||
# model — #433). An intrinsic routing prefix whose bare id is NOT a
|
||||
# first-party model of that namespace is kept whole, because the
|
||||
# proxy routes on the full string and truncating it 403s "model not
|
||||
# allowed": "bedrock/opus-4-6" stays intact (opus-4-6 ∉ bedrock
|
||||
# catalog — #3872).
|
||||
#
|
||||
# Unknown prefixes (e.g. "zai-org/GLM-5.1" on DeepInfra) are intrinsic
|
||||
# to the model ID and always preserved (#548). The redundant-prefix
|
||||
# strip that matches the *configured* provider's own family is handled
|
||||
# earlier by the ``prefix == config_provider`` branch.
|
||||
_cp_lower = (config_provider or "").strip().lower()
|
||||
_is_custom = _cp_lower == "custom" or _cp_lower.startswith("custom:")
|
||||
if prefix in _PROVIDER_MODELS and (
|
||||
not _is_custom or _is_first_party_model(prefix, bare)
|
||||
):
|
||||
return bare, config_provider, config_base_url
|
||||
# Unknown prefix (not a named provider) — pass full model_id through.
|
||||
# Intrinsic / unknown prefix — pass the full model_id through unchanged.
|
||||
return model_id, config_provider, config_base_url
|
||||
|
||||
# If prefix does NOT match config provider, the user picked a cross-provider model
|
||||
|
||||
@@ -159,6 +159,89 @@ def test_custom_provider_model_with_slash_routes_to_named_custom_provider():
|
||||
assert base_url == 'http://lmstudio.local:1234/v1'
|
||||
|
||||
|
||||
# ── #3872: bare ``custom`` provider is a vendor-routing proxy — preserve the
|
||||
# full model id (the prefix is intrinsic). #433's redundant-prefix strip is
|
||||
# scoped to real first-party providers (provider=openai + proxy base_url),
|
||||
# which is covered by test_custom_endpoint_slash_model_routes_to_custom_not_openrouter.
|
||||
|
||||
def test_custom_remote_preserves_intrinsic_vendor_prefix_3872():
|
||||
"""#3872: bedrock/opus-4-6 on a bare-custom remote proxy keeps its full id.
|
||||
|
||||
A bare ``custom`` provider with a remote base_url is a vendor-routing proxy
|
||||
(LiteLLM, Bedrock gateway). ``bedrock/`` is an intrinsic routing segment the
|
||||
proxy needs whole; stripping it to ``opus-4-6`` makes the proxy return 403
|
||||
"model not allowed for your group".
|
||||
"""
|
||||
model, provider, base_url = _resolve_with_config(
|
||||
'bedrock/opus-4-6',
|
||||
provider='custom',
|
||||
base_url='https://router.example.com/v1',
|
||||
)
|
||||
assert model == 'bedrock/opus-4-6', f"intrinsic prefix must be preserved, got {model!r}"
|
||||
assert provider == 'custom'
|
||||
assert base_url == 'https://router.example.com/v1'
|
||||
|
||||
|
||||
def test_custom_remote_strips_redundant_first_party_prefix_433():
|
||||
"""#433: bare-custom remote proxy still strips a REDUNDANT first-party prefix.
|
||||
|
||||
``gpt-5.4`` IS a first-party OpenAI model, so ``openai/`` is a redundant
|
||||
leftover and the proxy expects the bare id. This is the behaviour pinned by
|
||||
test_sprint40_ui_polish.py::test_prefixed_model_stripped_for_custom_endpoint;
|
||||
the #3872 fix must keep it while preserving intrinsic vendor prefixes.
|
||||
"""
|
||||
model, provider, base_url = _resolve_with_config(
|
||||
'openai/gpt-5.4',
|
||||
provider='custom',
|
||||
base_url='https://router.example.com/v1',
|
||||
)
|
||||
assert model == 'gpt-5.4', f"redundant first-party prefix must be stripped, got {model!r}"
|
||||
assert provider == 'custom'
|
||||
|
||||
|
||||
def test_custom_remote_preserves_unknown_prefix_548():
|
||||
"""#548: an unknown vendor prefix (zai-org/GLM-5.1) is always preserved."""
|
||||
model, provider, base_url = _resolve_with_config(
|
||||
'zai-org/GLM-5.1',
|
||||
provider='custom',
|
||||
base_url='https://api.deepinfra.com/v1/openai',
|
||||
)
|
||||
assert model == 'zai-org/GLM-5.1', f"unknown prefix must be preserved, got {model!r}"
|
||||
assert provider == 'custom'
|
||||
|
||||
|
||||
def test_named_custom_slug_preserves_intrinsic_vendor_prefix_3872():
|
||||
"""#3872 (named-custom variant): provider=custom:<slug> + remote base_url also
|
||||
preserves an intrinsic vendor prefix when the typed model isn't in the entry.
|
||||
|
||||
A model typed/selected that is not listed in the custom_providers[] entry
|
||||
falls through to the base_url branch; a bare id NOT first-party of the prefix
|
||||
(bedrock/opus-4-6) must still be kept whole, same as bare ``custom``.
|
||||
"""
|
||||
model, provider, base_url = _resolve_with_config(
|
||||
'bedrock/opus-4-6',
|
||||
provider='custom:my-gateway',
|
||||
base_url='https://router.example.com/v1',
|
||||
)
|
||||
assert model == 'bedrock/opus-4-6', f"intrinsic prefix must be preserved for custom:slug, got {model!r}"
|
||||
|
||||
|
||||
def test_first_party_provider_proxy_still_strips_prefix_433():
|
||||
"""#433/dc2334c5: provider=openai + remote proxy still strips the prefix.
|
||||
|
||||
This is the deliberate behaviour the #3872 fix must NOT regress: a real
|
||||
first-party provider pointed at an OpenAI-compatible proxy expects the bare
|
||||
id. (Mirrors the public-host branch of
|
||||
test_custom_endpoint_slash_model_routes_to_custom_not_openrouter.)
|
||||
"""
|
||||
model, provider, base_url = _resolve_with_config(
|
||||
'openai/gpt-5.4',
|
||||
provider='openai',
|
||||
base_url='https://litellm.example.com/v1',
|
||||
)
|
||||
assert model == 'gpt-5.4', f"redundant first-party prefix must be stripped, got {model!r}"
|
||||
|
||||
|
||||
def test_custom_provider_models_dict_routes_to_named_custom_provider():
|
||||
"""Models listed only under custom_providers[].models still route to that endpoint."""
|
||||
model, provider, base_url = _resolve_with_config(
|
||||
|
||||
Reference in New Issue
Block a user