CanGeng/llm-fallback

README 摘要

llm-fallback English 中文 Multi-level provider fallback for same-named models in the DeepSeek Harness. A pool maps one model id to an ordered provider list. The first provider is the primary; each later one is a fallback. The plugin keeps every step on the primary (stable request prefix → KV cache reuse) and only fails over when the current level is exhausted. Behavior - agent/request picks the pool level for each step. A new step starts at the primary (level 0) unless the explicitly requested provider is itself a pool member — then it starts there and fails over forward. A recovered primary is therefore reused immediately on the next step. It also probes each candidate with resolveCallConfig and skips (recording) any that would fail before streaming — a missing route ( NO ADAPTER ), an unknown model, or an unsupported reasoning effort. - agent/request-error records each level's failure, then advances the level by returning { kind: 'retry' } . When every level is exhausted it throws one aggregated error listing each level's failure ( provider: code — message ). Same-provider retries happen first (via dsh-llm-retry , which sits outside this plugin), so transient errors like RATE LIMIT…

分类