fix(onboarding): allow keyless setup for self-hosted providers (#1499 third sub-bug)

Pre-fix, the wizard rejected an empty api_key for every provider in
_SUPPORTED_PROVIDER_SETUPS — including lmstudio, ollama, and custom,
which run keyless on the vast majority of local installs. The agent's
LMSTUDIO_NOAUTH_PLACEHOLDER substitution at chat-time was the workaround
for the no-auth case, but the wizard side rejected the empty input first.
Users had to type random gibberish into the API key field to clear the
form — the third sub-bug from #1420 that the prior commit's PR description
explicitly punted to a follow-up.

Surfaced by Nathan during PR review: "I think it's too weird for users
to have to type a string into the API key field, right?"  Yes — and the
probe (#1499) makes the cleanest fix strictly better: we accept empty
keys, and the probe gives instant feedback ("Connected. 2 model(s)
available." for keyless servers, "401" for auth-required servers).

Backend changes
---------------

* `api/onboarding.py` — `_SUPPORTED_PROVIDER_SETUPS` gains
  `key_optional: True` for `lmstudio`, `ollama`, `custom`. Cloud
  providers (openrouter, anthropic, openai, gemini, deepseek, …)
  remain key_required.

* `apply_onboarding_setup` skips the "{env_var} is required" check
  when `key_optional` is set AND no key is supplied. No write to .env
  for the empty-key case (no `LM_API_KEY=*** placeholder lying in the
  user's .env`).

* `_status_from_runtime` reports `provider_ready=True` for key_optional
  providers based on `requires_base_url` alone, so the wizard doesn't
  refire on the next page load just because there's no api_key. Cloud
  providers still need a key for provider_ready=True.

* `_build_setup_catalog` exposes the `key_optional` flag to the frontend.

Frontend changes
----------------

* `static/onboarding.js` — new `_renderOnboardingApiKeyField()` helper.
  For key_optional providers:
    - Label: "API key (optional)"
    - Placeholder: "Leave blank for keyless servers"
    - Inline italic muted help: "Most LM Studio / Ollama / vLLM installs
      run keyless — leave this blank if your server doesn't require
      authentication. Use the Test connection button to verify."
  For cloud providers: unchanged (label "API key", standard placeholder,
  no help block).

* The api-key input also now triggers `_scheduleOnboardingProbe()` on
  oninput, so changing the key re-runs the probe — handles "the server
  rejected my empty key with 401, let me add one and retry."

* `static/i18n.js` — 3 new keys × 9 locales (canonical English in `en`,
  English fallback with `// TODO: translate` markers in the other 8).

* `static/style.css` — `.onboarding-api-key-help` rule for the muted
  italic helper paragraph.

Verified end-to-end on port 8789
--------------------------------

Spun up an isolated test server + a mock LM Studio at
`127.0.0.1:11234/v1/models`. Stepped through the wizard:

* Picked LM Studio → field label flipped to "API key (optional)",
  placeholder showed "Leave blank for keyless servers", help text
  rendered in italic muted gray below.
* Switched to Anthropic → label reverted to "API key", help text
  disappeared. Visual hierarchy correct.
* Left api_key blank, set base_url to the mock, clicked Test connection
  → green "Connected. 2 model(s) available." banner. Probe-discovered
  models populated the workspace-step dropdown.
* Continued through to the finish step. config.yaml written with
  provider/model/base_url. **`.env` does NOT exist** — no placeholder
  string written. `chat_ready: true`, `state: ready`.
* Vision tool confirmed the visual hierarchy: subtle italic help
  reads as documentation, prominent green banner pops as status.

Tests
-----

`tests/test_issue1499_keyless_onboarding.py` — 16 tests in 3 classes:

  TestKeyOptionalProviderSchema (5)
    - lmstudio / ollama / custom declare key_optional=True
    - openrouter / anthropic / openai do NOT (regression defense)
    - setup catalog exposes the flag

  TestKeylessOnboarding (6)
    - lmstudio / ollama / custom: empty api_key accepted, no .env write
    - openrouter / anthropic: empty api_key still rejected
    - lmstudio with explicit key still writes .env (regression defense)

  TestKeylessChatReady (5)
    - lmstudio / ollama: provider_ready=True with no key
    - custom: provider_ready=True with key+base_url, False without base_url
    - openrouter: provider_ready=False with no key (regression defense)
    - End-to-end get_onboarding_status reports chat_ready=True

Full suite: 3901 → 3917 passing (+16 from this commit; +22 cumulative
from the PR's earlier commit). 0 failures.

Closes #1499 (all three sub-bugs from #1420 now addressed)
This commit is contained in:
Hermes Bot
2026-05-03 03:07:07 +00:00
parent 8616033605
commit 8f4692b8cf
5 changed files with 479 additions and 19 deletions

View File

@@ -70,6 +70,10 @@ _SUPPORTED_PROVIDER_SETUPS = {
"default_model": "qwen3:32b",
"default_base_url": "http://localhost:11434/v1",
"requires_base_url": True,
# Local Ollama runs keyless by default — only Ollama Cloud requires
# OLLAMA_API_KEY. The wizard accepts an empty api_key for this
# provider; users with auth enabled can still type one. See #1499.
"key_optional": True,
"models": [],
"category": "self_hosted",
},
@@ -90,6 +94,11 @@ _SUPPORTED_PROVIDER_SETUPS = {
"default_model": "gpt-4o-mini",
"default_base_url": "http://localhost:1234/v1",
"requires_base_url": True,
# Most LM Studio installs run keyless (LMSTUDIO_NOAUTH_PLACEHOLDER on the
# agent side handles this). The wizard accepts an empty api_key; auth-
# enabled servers still need one but the user types it in the same field.
# See #1499 (third sub-bug from #1420).
"key_optional": True,
"models": [],
"category": "self_hosted",
},
@@ -98,6 +107,11 @@ _SUPPORTED_PROVIDER_SETUPS = {
"env_var": "OPENAI_API_KEY",
"default_model": "gpt-4o-mini",
"requires_base_url": True,
# Many self-hosted OpenAI-compatible servers (vLLM, llama-server,
# TabbyAPI, etc.) run keyless behind a private network. The wizard
# accepts an empty api_key — auth-protected endpoints can still
# supply one. See #1499.
"key_optional": True,
"models": [],
"category": "self_hosted",
},
@@ -574,12 +588,30 @@ def _status_from_runtime(cfg: dict, imports_ok: bool) -> dict:
provider_ready = False
if provider_configured:
if provider == "custom":
provider_ready = bool(
base_url and _provider_api_key_present(provider, cfg, env_values)
)
elif provider in _SUPPORTED_PROVIDER_SETUPS:
provider_ready = _provider_api_key_present(provider, cfg, env_values)
meta = _SUPPORTED_PROVIDER_SETUPS.get(provider, {})
if provider in _SUPPORTED_PROVIDER_SETUPS:
# key_optional providers (lmstudio, ollama, custom) are ready as
# soon as the user has saved a provider+model+base_url; an api_key
# is allowed but not required. The agent runtime substitutes a
# placeholder for keyless local servers (LMSTUDIO_NOAUTH_PLACEHOLDER
# for lmstudio, equivalent paths for ollama / custom). See #1499
# third sub-bug from #1420.
if meta.get("key_optional"):
if meta.get("requires_base_url"):
provider_ready = bool(base_url)
else:
provider_ready = True
else:
# Standard wizard provider (openrouter, anthropic, openai, gemini,
# deepseek, zai, …) — needs an api_key. Custom historically also
# took this branch, but is now key_optional via the meta flag.
if meta.get("requires_base_url"):
provider_ready = bool(
base_url
and _provider_api_key_present(provider, cfg, env_values)
)
else:
provider_ready = _provider_api_key_present(provider, cfg, env_values)
else:
# Unknown provider — may be an OAuth flow (openai-codex, copilot, etc.)
# OR an API-key provider not in the quick-setup list (minimax-cn, deepseek,
@@ -655,6 +687,10 @@ def _build_setup_catalog(cfg: dict) -> dict:
"default_model": meta["default_model"],
"default_base_url": meta.get("default_base_url") or "",
"requires_base_url": bool(meta.get("requires_base_url")),
# #1499 (third sub-bug from #1420) — providers that may run
# keyless (lmstudio, ollama, custom). Frontend uses this to
# show a "(optional)" hint and allow Continue without a key.
"key_optional": bool(meta.get("key_optional")),
"models": list(meta.get("models", [])),
"category": meta.get("category", "easy_start"),
"quick": meta.get("quick", False),
@@ -842,7 +878,14 @@ def apply_onboarding_setup(body: dict) -> dict:
env_values = _load_env_file(env_path)
if not api_key and not _provider_api_key_present(provider, cfg, env_values):
raise ValueError(f"{provider_meta['env_var']} is required")
# Providers that may run keyless (lmstudio, ollama, custom — gated by
# `key_optional` in _SUPPORTED_PROVIDER_SETUPS) are allowed to onboard
# with no api_key. The agent runtime substitutes a placeholder
# (LMSTUDIO_NOAUTH_PLACEHOLDER) for those, and the probe (#1499) gives
# the user immediate feedback if their server actually does require
# auth (http_4xx with status 401). See #1499 third sub-bug from #1420.
if not provider_meta.get("key_optional"):
raise ValueError(f"{provider_meta['env_var']} is required")
model_cfg = cfg.get("model", {})
if not isinstance(model_cfg, dict):

View File

@@ -622,6 +622,9 @@ const LOCALES = {
provider_category_specialized: 'Specialized',
onboarding_api_key_label: 'API key',
onboarding_api_key_placeholder: 'Leave blank to keep an existing saved key',
onboarding_api_key_label_optional: 'API key (optional)',
onboarding_api_key_placeholder_optional: 'Leave blank for keyless servers',
onboarding_api_key_help_keyless: 'Most LM Studio / Ollama / vLLM installs run keyless — leave this blank if your server doesn\'t require authentication. Use the Test connection button to verify.',
onboarding_api_key_help_prefix: 'Saved as a secret in your Hermes .env file using',
onboarding_base_url_label: 'Base URL',
onboarding_base_url_placeholder: 'https://your-endpoint.example/v1',
@@ -1515,6 +1518,9 @@ const LOCALES = {
provider_category_specialized: '専門用途',
onboarding_api_key_label: 'APIキー',
onboarding_api_key_placeholder: '空欄で既存の保存済みキーを維持',
onboarding_api_key_label_optional: 'API key (optional)', // TODO: translate
onboarding_api_key_placeholder_optional: 'Leave blank for keyless servers', // TODO: translate
onboarding_api_key_help_keyless: 'Most LM Studio / Ollama / vLLM installs run keyless — leave this blank if your server doesn\'t require authentication. Use the Test connection button to verify.', // TODO: translate
onboarding_api_key_help_prefix: 'Hermes の .env ファイルにシークレットとして保存されます — 使用変数:',
onboarding_base_url_label: 'ベース URL',
onboarding_base_url_placeholder: 'https://your-endpoint.example/v1',
@@ -2208,6 +2214,9 @@ const LOCALES = {
provider_category_specialized: 'Специализированные',
onboarding_api_key_label: 'Ключ API',
onboarding_api_key_placeholder: 'Оставьте пустым, чтобы сохранить уже сохранённый ключ',
onboarding_api_key_label_optional: 'API key (optional)', // TODO: translate
onboarding_api_key_placeholder_optional: 'Leave blank for keyless servers', // TODO: translate
onboarding_api_key_help_keyless: 'Most LM Studio / Ollama / vLLM installs run keyless — leave this blank if your server doesn\'t require authentication. Use the Test connection button to verify.', // TODO: translate
oauth_login_codex: 'Login with Codex (ChatGPT)', // TODO: translate
oauth_codex_step1: 'Step 1: Visit this URL and enter the code', // TODO: translate
oauth_codex_step2: 'Step 2: Enter this code on the page', // TODO: translate
@@ -3040,6 +3049,9 @@ const LOCALES = {
provider_category_specialized: 'Especializados',
onboarding_api_key_label: 'API key',
onboarding_api_key_placeholder: 'Déjala en blanco para conservar una key ya guardada',
onboarding_api_key_label_optional: 'API key (optional)', // TODO: translate
onboarding_api_key_placeholder_optional: 'Leave blank for keyless servers', // TODO: translate
onboarding_api_key_help_keyless: 'Most LM Studio / Ollama / vLLM installs run keyless — leave this blank if your server doesn\'t require authentication. Use the Test connection button to verify.', // TODO: translate
oauth_login_codex: 'Login with Codex (ChatGPT)', // TODO: translate
oauth_codex_step1: 'Step 1: Visit this URL and enter the code', // TODO: translate
oauth_codex_step2: 'Step 2: Enter this code on the page', // TODO: translate
@@ -4010,6 +4022,9 @@ const LOCALES = {
provider_category_specialized: 'Spezialisiert',
onboarding_api_key_label: 'API-Schlüssel',
onboarding_api_key_placeholder: 'sk-…',
onboarding_api_key_label_optional: 'API key (optional)', // TODO: translate
onboarding_api_key_placeholder_optional: 'Leave blank for keyless servers', // TODO: translate
onboarding_api_key_help_keyless: 'Most LM Studio / Ollama / vLLM installs run keyless — leave this blank if your server doesn\'t require authentication. Use the Test connection button to verify.', // TODO: translate
oauth_login_codex: 'Login with Codex (ChatGPT)', // TODO: translate
oauth_codex_step1: 'Step 1: Visit this URL and enter the code', // TODO: translate
oauth_codex_step2: 'Step 2: Enter this code on the page', // TODO: translate
@@ -4696,6 +4711,9 @@ const LOCALES = {
provider_category_specialized: '专业服务',
onboarding_api_key_label: 'API key',
onboarding_api_key_placeholder: '留空可保留已保存的 key',
onboarding_api_key_label_optional: 'API key (optional)', // TODO: translate
onboarding_api_key_placeholder_optional: 'Leave blank for keyless servers', // TODO: translate
onboarding_api_key_help_keyless: 'Most LM Studio / Ollama / vLLM installs run keyless — leave this blank if your server doesn\'t require authentication. Use the Test connection button to verify.', // TODO: translate
oauth_login_codex: 'Login with Codex (ChatGPT)', // TODO: translate
oauth_codex_step1: 'Step 1: Visit this URL and enter the code', // TODO: translate
oauth_codex_step2: 'Step 2: Enter this code on the page', // TODO: translate
@@ -5494,6 +5512,9 @@ const LOCALES = {
onboarding_api_key_help_prefix: '\u900f\u904e\u4ee5\u4e0b\u65b9\u5f0f\u5132\u5b58\u70ba Hermes .env \u6a94\u6848\u4e2d\u7684\u6a5f\u5bc6',
onboarding_api_key_label: 'API \u91d1\u9470',
onboarding_api_key_placeholder: '\u7559\u7a7a\u4ee5\u4fdd\u7559\u5df2\u5132\u5b58\u7684\u91d1\u9470',
onboarding_api_key_label_optional: 'API key (optional)', // TODO: translate
onboarding_api_key_placeholder_optional: 'Leave blank for keyless servers', // TODO: translate
onboarding_api_key_help_keyless: 'Most LM Studio / Ollama / vLLM installs run keyless — leave this blank if your server doesn\'t require authentication. Use the Test connection button to verify.', // TODO: translate
onboarding_back: '\u4e0a\u4e00\u6b65',
onboarding_badge: '\u9996\u6b21\u57f7\u884c',
onboarding_base_url_help: '\u7528\u65bc OpenAI \u76f8\u5bb9\u8def\u7531\u5668\u3001\u81ea\u67b6\u4f3a\u670d\u5668\u3001LiteLLM\u3001Ollama\u3001LM Studio\u3001vLLM \u7b49\u7aef\u9ede\u3002',
@@ -6532,6 +6553,9 @@ const LOCALES = {
oauth_codex_error: 'OAuth login failed', // TODO: translate
oauth_codex_expired: 'Code expired, please try again', // TODO: translate
onboarding_api_key_placeholder: 'Deixe em branco para manter key existente',
onboarding_api_key_label_optional: 'API key (optional)', // TODO: translate
onboarding_api_key_placeholder_optional: 'Leave blank for keyless servers', // TODO: translate
onboarding_api_key_help_keyless: 'Most LM Studio / Ollama / vLLM installs run keyless — leave this blank if your server doesn\'t require authentication. Use the Test connection button to verify.', // TODO: translate
onboarding_api_key_help_prefix: 'Salvo como segredo no .env do Hermes usando',
onboarding_base_url_label: 'Base URL',
onboarding_base_url_placeholder: 'https://seu-endpoint.exemplo/v1',
@@ -7331,6 +7355,9 @@ const LOCALES = {
oauth_codex_error: 'OAuth login failed', // TODO: translate
oauth_codex_expired: 'Code expired, please try again', // TODO: translate
onboarding_api_key_placeholder: 'Leave blank to keep an existing saved key',
onboarding_api_key_label_optional: 'API key (optional)', // TODO: translate
onboarding_api_key_placeholder_optional: 'Leave blank for keyless servers', // TODO: translate
onboarding_api_key_help_keyless: 'Most LM Studio / Ollama / vLLM installs run keyless — leave this blank if your server doesn\'t require authentication. Use the Test connection button to verify.', // TODO: translate
onboarding_api_key_help_prefix: 'Saved as a secret in your Hermes .env file using',
onboarding_base_url_label: 'Base URL',
onboarding_base_url_placeholder: 'https://your-endpoint.example/v1',

View File

@@ -185,6 +185,21 @@ function _renderOnboardingBaseUrlField(showBaseUrl){
return `<label class="onboarding-field"><span>${t('onboarding_base_url_label')}</span><input id="onboardingBaseUrlInput" value="${esc(ONBOARDING.form.baseUrl||'')}" placeholder="${t('onboarding_base_url_placeholder')}" oninput="ONBOARDING.form.baseUrl=this.value;_scheduleOnboardingProbe()" onblur="_runOnboardingProbe()"></label><div class="onboarding-probe-row"><button type="button" class="onboarding-probe-btn" ${testBtnDisabled} onclick="_runOnboardingProbe({force:true})">${esc(testBtnLabel)}</button></div>${banner}`;
}
function _renderOnboardingApiKeyField(){
// Renders the API-key input. For providers flagged `key_optional` in the
// setup catalog (lmstudio, ollama, custom — typically self-hosted servers
// that run keyless by default), the field shows an "(optional)" hint and
// empty input is accepted on Continue. Pre-#1499-third-sub-bug-fix the
// wizard required a non-empty string here even for keyless installs, which
// forced users to type random gibberish to clear onboarding.
const provider=_getOnboardingSetupProvider(ONBOARDING.form.provider);
const keyOptional=!!(provider&&provider.key_optional);
const labelKey=keyOptional?'onboarding_api_key_label_optional':'onboarding_api_key_label';
const placeholderKey=keyOptional?'onboarding_api_key_placeholder_optional':'onboarding_api_key_placeholder';
const helpHtml=keyOptional?`<p class="onboarding-copy onboarding-api-key-help">${esc(t('onboarding_api_key_help_keyless')||'')}</p>`:'';
return `<label class="onboarding-field" id="onboardingApiKeyField"><span>${t(labelKey)}</span><input id="onboardingApiKeyInput" type="password" value="${esc(ONBOARDING.form.apiKey||'')}" placeholder="${t(placeholderKey)}" oninput="ONBOARDING.form.apiKey=this.value;_scheduleOnboardingProbe()"></label>${helpHtml}`;
}
function _getOnboardingSelectedModel(){
return ONBOARDING.form.model||'';
}
@@ -265,10 +280,7 @@ function _renderOnboardingBody(){
<span>${t('onboarding_provider_label')}</span>
<select id="onboardingProviderSelect" onchange="syncOnboardingProvider(this.value)">${groupedOptions}</select>
</label>
<label class="onboarding-field" id="onboardingApiKeyField">
<span>${t('onboarding_api_key_label')}</span>
<input id="onboardingApiKeyInput" type="password" value="${esc(ONBOARDING.form.apiKey||'')}" placeholder="${t('onboarding_api_key_placeholder')}" oninput="ONBOARDING.form.apiKey=this.value">
</label>
${_renderOnboardingApiKeyField()}
${_renderOnboardingBaseUrlField(showBaseUrl)}
<p class="onboarding-copy">${keyHelp}</p>`;
} else {
@@ -286,10 +298,7 @@ function _renderOnboardingBody(){
<span>${t('onboarding_provider_label')}</span>
<select id="onboardingProviderSelect" onchange="syncOnboardingProvider(this.value)">${groupedOptions}</select>
</label>
<label class="onboarding-field" id="onboardingApiKeyField">
<span>${t('onboarding_api_key_label')}</span>
<input id="onboardingApiKeyInput" type="password" value="${esc(ONBOARDING.form.apiKey||'')}" placeholder="${t('onboarding_api_key_placeholder')}" oninput="ONBOARDING.form.apiKey=this.value">
</label>
${_renderOnboardingApiKeyField()}
${_renderOnboardingBaseUrlField(showBaseUrl)}
<p class="onboarding-copy">${keyHelp}</p>`;
}
@@ -302,10 +311,7 @@ function _renderOnboardingBody(){
<span>${t('onboarding_provider_label')}</span>
<select id="onboardingProviderSelect" onchange="syncOnboardingProvider(this.value)">${groupedOptions}</select>
</label>
<label class="onboarding-field">
<span>${t('onboarding_api_key_label')}</span>
<input id="onboardingApiKeyInput" type="password" value="${esc(ONBOARDING.form.apiKey||'')}" placeholder="${t('onboarding_api_key_placeholder')}" oninput="ONBOARDING.form.apiKey=this.value">
</label>
${_renderOnboardingApiKeyField()}
${_renderOnboardingBaseUrlField(showBaseUrl)}
<p class="onboarding-copy">${keyHelp}</p>
<div class="onboarding-oauth-card" id="codexOAuthCard">

View File

@@ -497,6 +497,9 @@
.onboarding-probe-banner.onboarding-probe-ok{border-color:#1f8a4f;color:#9ce0b4;background:rgba(31,138,79,.12);}
.onboarding-probe-banner.onboarding-probe-probing{color:var(--muted);}
.onboarding-probe-banner.onboarding-probe-error{border-color:#9c3a3a;color:#f5a3a3;background:rgba(156,58,58,.12);}
/* Onboarding api-key keyless hint (#1499 third sub-bug) — subtle muted text */
/* sized below the input so it reads as inline help, not a banner. */
.onboarding-api-key-help{margin:4px 0 0;font-size:11px;color:var(--muted);font-style:italic;}
.onboarding-summary{display:grid;grid-template-columns:repeat(auto-fit,minmax(180px,1fr));gap:12px;}
.onboarding-summary div{padding:14px;border-radius:14px;background:rgba(255,255,255,.03);border:1px solid var(--border);display:flex;flex-direction:column;gap:5px;}
.onboarding-summary strong{font-size:12px;letter-spacing:.04em;text-transform:uppercase;color:var(--muted);}

View File

@@ -0,0 +1,381 @@
"""Regression: self-hosted providers accept empty api_key in onboarding (#1499 sub-bug 3).
Pre-fix, ``apply_onboarding_setup`` rejected an empty ``api_key`` for every
wizard provider with the error ``f"{env_var} is required"``. For LM Studio,
Ollama, and Custom — which run keyless on most local installs — this forced
users to type a placeholder string into the API key field just to clear the
wizard. The ``LMSTUDIO_NOAUTH_PLACEHOLDER`` substitution at chat-time was the
agent's workaround for the no-auth case, but the wizard side rejected the
empty input first, so users never got that far without typing gibberish.
The fix adds a ``key_optional: True`` flag to the affected providers in
``_SUPPORTED_PROVIDER_SETUPS``. When that flag is set:
* ``apply_onboarding_setup`` skips the "key required" check.
* No write to ``.env`` happens for the empty-key case (no
``LM_API_KEY=*** placeholder lying in the user's .env file`` either).
* ``_status_from_runtime`` reports ``provider_ready=True`` based on
``base_url`` alone, so the wizard doesn't refire on the next page load
just because there's no api_key.
* The setup catalog exposes ``key_optional`` so the frontend can render
"(optional)" hint copy + accept empty submit.
Symmetric tests verify the existing required-key path still rejects empty
api_keys for cloud providers (openrouter, anthropic, openai), so this fix
doesn't accidentally make every provider keyless.
Reporters: @chwps, @AdoneyGalvan via #1420 → split into #1499.
"""
from __future__ import annotations
import sys
import types
from pathlib import Path
import pytest
import api.config as config
import api.profiles as profiles
def _install_fake_hermes_cli(monkeypatch):
"""Stub hermes_cli modules so tests are deterministic and offline.
Mirrors the helper in test_provider_management.py — kept inline so this
regression test stays self-contained.
"""
fake_pkg = types.ModuleType("hermes_cli")
fake_pkg.__path__ = []
fake_models = types.ModuleType("hermes_cli.models")
fake_models.list_available_providers = lambda: []
fake_models.provider_model_ids = lambda pid: []
fake_auth = types.ModuleType("hermes_cli.auth")
fake_auth.get_auth_status = lambda _pid: {}
monkeypatch.setitem(sys.modules, "hermes_cli", fake_pkg)
monkeypatch.setitem(sys.modules, "hermes_cli.models", fake_models)
monkeypatch.setitem(sys.modules, "hermes_cli.auth", fake_auth)
def _isolate_onboarding_writes(monkeypatch, tmp_path):
"""Redirect every onboarding write target to ``tmp_path`` and clear
every relevant env var so the test starts from a known clean state.
Pattern from webui-onboarding-provider-readiness skill — without this,
tests that call ``apply_onboarding_setup`` directly write to the real
``~/.hermes`` and clobber the developer's actual config.
"""
from api import onboarding as ob
monkeypatch.setattr(ob, "_get_active_hermes_home", lambda: tmp_path)
cfg_path = tmp_path / "config.yaml"
monkeypatch.setattr(ob, "_get_config_path", lambda: cfg_path)
monkeypatch.setattr(profiles, "get_active_hermes_home", lambda: tmp_path)
monkeypatch.delenv("HERMES_WEBUI_SKIP_ONBOARDING", raising=False)
for var in (
"LM_API_KEY", "LMSTUDIO_API_KEY", "OLLAMA_API_KEY", "OPENAI_API_KEY",
"OPENROUTER_API_KEY", "ANTHROPIC_API_KEY", "GOOGLE_API_KEY",
"GH_TOKEN", "GITHUB_TOKEN",
):
monkeypatch.delenv(var, raising=False)
return cfg_path
class TestKeyOptionalProviderSchema:
"""The catalog declares which providers may run keyless."""
def test_lmstudio_is_key_optional(self):
from api.onboarding import _SUPPORTED_PROVIDER_SETUPS
assert _SUPPORTED_PROVIDER_SETUPS["lmstudio"].get("key_optional") is True, (
"lmstudio must declare key_optional=True so onboarding accepts an "
"empty api_key. Pre-fix the wizard required users to type "
"gibberish to clear the form. See #1499 (third sub-bug from #1420)."
)
def test_ollama_is_key_optional(self):
from api.onboarding import _SUPPORTED_PROVIDER_SETUPS
assert _SUPPORTED_PROVIDER_SETUPS["ollama"].get("key_optional") is True, (
"ollama must declare key_optional=True — local Ollama runs keyless "
"by default."
)
def test_custom_is_key_optional(self):
from api.onboarding import _SUPPORTED_PROVIDER_SETUPS
assert _SUPPORTED_PROVIDER_SETUPS["custom"].get("key_optional") is True, (
"custom must declare key_optional=True — many self-hosted "
"OpenAI-compatible servers (vLLM, llama-server, TabbyAPI) run "
"keyless behind a private network."
)
def test_cloud_providers_are_not_key_optional(self):
"""Regression-defense: openrouter/anthropic/openai must STILL require a key."""
from api.onboarding import _SUPPORTED_PROVIDER_SETUPS
for pid in ("openrouter", "anthropic", "openai"):
assert not _SUPPORTED_PROVIDER_SETUPS[pid].get("key_optional"), (
f"{pid} must NOT be key_optional — cloud providers always need "
f"a real key. This test catches an accidental flag flip."
)
def test_setup_catalog_exposes_key_optional_flag(self):
"""Frontend reads `provider.key_optional` from the catalog."""
from api.onboarding import _build_setup_catalog
catalog = _build_setup_catalog({"model": {"provider": "lmstudio"}})
by_id = {p["id"]: p for p in catalog["providers"]}
assert by_id["lmstudio"]["key_optional"] is True
assert by_id["ollama"]["key_optional"] is True
assert by_id["custom"]["key_optional"] is True
assert by_id["openrouter"]["key_optional"] is False, (
"Catalog must expose key_optional=False for cloud providers so "
"the frontend doesn't accidentally label them optional."
)
class TestKeylessOnboarding:
"""``apply_onboarding_setup`` accepts empty api_key for key_optional providers."""
def test_lmstudio_empty_api_key_accepted(self, monkeypatch, tmp_path):
"""Pre-fix this raised; post-fix it succeeds and writes no .env entry."""
_install_fake_hermes_cli(monkeypatch)
cfg_path = _isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
# Empty api_key — should NOT raise.
ob.apply_onboarding_setup({
"provider": "lmstudio",
"model": "qwen3-27b",
"base_url": "http://example.local:1234/v1",
"api_key": "",
})
# config.yaml gets written with provider/model/base_url
assert cfg_path.exists()
cfg_text = cfg_path.read_text(encoding="utf-8")
assert "provider: lmstudio" in cfg_text
assert "base_url: http://example.local:1234/v1" in cfg_text
# .env should NOT have an API_KEY entry — empty key means we don't
# write a placeholder into .env.
env_path = tmp_path / ".env"
if env_path.exists():
env_text = env_path.read_text(encoding="utf-8")
assert "LM_API_KEY=" not in env_text, (
f"Onboarding wrote LM_API_KEY to .env even though user "
f"submitted an empty api_key. .env contents:\n{env_text}"
)
assert "LMSTUDIO_API_KEY=" not in env_text
def test_ollama_empty_api_key_accepted(self, monkeypatch, tmp_path):
_install_fake_hermes_cli(monkeypatch)
_isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
ob.apply_onboarding_setup({
"provider": "ollama",
"model": "qwen3:32b",
"base_url": "http://localhost:11434/v1",
"api_key": "",
})
env_path = tmp_path / ".env"
if env_path.exists():
assert "OLLAMA_API_KEY=" not in env_path.read_text(encoding="utf-8")
def test_custom_empty_api_key_accepted(self, monkeypatch, tmp_path):
_install_fake_hermes_cli(monkeypatch)
_isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
ob.apply_onboarding_setup({
"provider": "custom",
"model": "gpt-4o-mini",
"base_url": "http://my-vllm.local/v1",
"api_key": "",
})
env_path = tmp_path / ".env"
if env_path.exists():
assert "OPENAI_API_KEY=" not in env_path.read_text(encoding="utf-8")
def test_openrouter_empty_api_key_still_rejected(self, monkeypatch, tmp_path):
"""Cloud providers must still reject empty api_key (regression defense)."""
_install_fake_hermes_cli(monkeypatch)
_isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
with pytest.raises(ValueError, match="OPENROUTER_API_KEY is required"):
ob.apply_onboarding_setup({
"provider": "openrouter",
"model": "anthropic/claude-sonnet-4.6",
"base_url": "",
"api_key": "",
})
def test_anthropic_empty_api_key_still_rejected(self, monkeypatch, tmp_path):
_install_fake_hermes_cli(monkeypatch)
_isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
with pytest.raises(ValueError, match="ANTHROPIC_API_KEY is required"):
ob.apply_onboarding_setup({
"provider": "anthropic",
"model": "claude-sonnet-4.6",
"api_key": "",
})
def test_lmstudio_with_explicit_api_key_still_writes_env(self, monkeypatch, tmp_path):
"""Auth-enabled LM Studio: user supplies a key, .env still gets written.
Regression-defense for the keyless path: when the user DOES supply a
key, we still write it under the canonical name (LM_API_KEY, post-#1500).
"""
_install_fake_hermes_cli(monkeypatch)
_isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
ob.apply_onboarding_setup({
"provider": "lmstudio",
"model": "qwen3-27b",
"base_url": "http://example.local:1234/v1",
"api_key": "real-secret-token",
})
env_text = (tmp_path / ".env").read_text(encoding="utf-8")
assert "LM_API_KEY=" in env_text, (
f"Auth-enabled lmstudio user supplied an api_key but .env doesn't "
f"contain LM_API_KEY. Contents:\n{env_text}"
)
class TestKeylessChatReady:
"""``provider_ready`` and ``chat_ready`` are True for key_optional providers."""
def test_lmstudio_keyless_provider_ready_via_status_runtime(
self, monkeypatch, tmp_path,
):
"""``_status_from_runtime`` returns provider_ready=True with no api_key."""
_install_fake_hermes_cli(monkeypatch)
_isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
cfg = {
"model": {
"provider": "lmstudio",
"default": "qwen3-27b",
"base_url": "http://example.local:1234/v1",
},
}
status = ob._status_from_runtime(cfg, imports_ok=True)
assert status.get("provider_ready") is True, (
"lmstudio with base_url + model + NO api_key must be provider_ready=True. "
"Otherwise the wizard refires on every page load even though the "
"user finished setup. See #1499 third sub-bug from #1420."
)
def test_ollama_keyless_provider_ready_via_status_runtime(
self, monkeypatch, tmp_path,
):
_install_fake_hermes_cli(monkeypatch)
_isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
cfg = {
"model": {
"provider": "ollama",
"default": "qwen3:32b",
"base_url": "http://localhost:11434/v1",
},
}
status = ob._status_from_runtime(cfg, imports_ok=True)
assert status.get("provider_ready") is True
def test_custom_keyless_provider_ready_requires_base_url(
self, monkeypatch, tmp_path,
):
"""custom is key_optional but still requires base_url."""
_install_fake_hermes_cli(monkeypatch)
_isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
# With base_url → ready
cfg_with = {
"model": {
"provider": "custom",
"default": "gpt-4o-mini",
"base_url": "http://my-vllm.local/v1",
},
}
assert ob._status_from_runtime(cfg_with, imports_ok=True).get("provider_ready") is True
# Without base_url → NOT ready (custom still requires it)
cfg_without = {
"model": {
"provider": "custom",
"default": "gpt-4o-mini",
},
}
assert ob._status_from_runtime(cfg_without, imports_ok=True).get("provider_ready") is False, (
"custom is key_optional but still requires base_url — this test "
"catches a regression where the requires_base_url check is "
"accidentally dropped for key_optional providers."
)
def test_openrouter_keyless_provider_ready_is_false(self, monkeypatch, tmp_path):
"""Cloud provider with no key → provider_ready=False (regression defense)."""
_install_fake_hermes_cli(monkeypatch)
_isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
cfg = {
"model": {
"provider": "openrouter",
"default": "anthropic/claude-sonnet-4.6",
},
}
assert ob._status_from_runtime(cfg, imports_ok=True).get("provider_ready") is False, (
"openrouter with no api_key must NOT be provider_ready — that "
"would silently let the wizard finish without the user actually "
"entering a key."
)
def test_lmstudio_keyless_chat_ready_via_full_status(self, monkeypatch, tmp_path):
"""End-to-end: get_onboarding_status reports chat_ready=True after keyless save."""
_install_fake_hermes_cli(monkeypatch)
cfg_path = _isolate_onboarding_writes(monkeypatch, tmp_path)
from api import onboarding as ob
ob.apply_onboarding_setup({
"provider": "lmstudio",
"model": "qwen3-27b",
"base_url": "http://example.local:1234/v1",
"api_key": "",
})
# Reload config so get_onboarding_status sees the just-written values.
# _swap_in_test_config-style — replicate just enough of that pattern.
old_cfg = dict(config.cfg)
old_mtime = config._cfg_mtime
config.cfg.clear()
try:
import yaml
config.cfg.update(yaml.safe_load(cfg_path.read_text(encoding="utf-8")) or {})
except Exception:
pass
try:
config._cfg_mtime = cfg_path.stat().st_mtime
except Exception:
config._cfg_mtime = 0.0
try:
status = ob.get_onboarding_status()
system = status.get("system", {})
assert system.get("provider_ready") is True, (
f"After saving lmstudio keyless config, provider_ready must be "
f"True. Got: provider_ready={system.get('provider_ready')!r}, "
f"chat_ready={system.get('chat_ready')!r}."
)
# chat_ready additionally requires _HERMES_FOUND + imports_ok which
# depend on the test environment; provider_ready is the bit this
# PR's fix actually controls. But if hermes is importable, it
# should also be chat_ready.
if system.get("imports_ok"):
assert system.get("chat_ready") is True
finally:
config.cfg.clear()
config.cfg.update(old_cfg)
config._cfg_mtime = old_mtime