TabbyAPITabbyAPI
TabbyAPI：ExLlamaV3 的官方 API 后端服务器（AGPL-3.0，1,502★ 推送 2026-10-30）—— FastAPI 本地推理服务，OpenAI 兼容端点，官方 GHCR 镜像覆盖 CUDA 12.8/13 与 AMD ROCm。 两条硬口径：README 官方警告「本仓是唯一官方信息源」——tabbyapi.com 反而非官方； rolling release、GitHub releases 为 0——变更记录线看 commits。
TabbyAPI: the official API backend server for ExLlamaV3 (AGPL-3.0, 1,502 stars, pushed 2026-10-30) — a FastAPI local inference server with OpenAI-compatible endpoints and official GHCR images covering CUDA 12.8/13 and AMD ROCm. Two hard wordings: the README's official warning says "the ONLY official source for TabbyAPI information is this repository" — tabbyapi.com is actually unofficial; rolling release with zero GitHub releases — the change-record line is commits.
适合与不适合When to use it, when not to
适合：ExLlamaV3（GPTQ 系）要 OpenAI 兼容 API；Docker 一键起本地推理服务。 不适合：生产级服务化（官方自述非生产向）；GGUF 全量化档广覆盖路线（llama.cpp 根）。
Fits: the ExLlamaV3 (GPTQ family) wanting an OpenAI-compatible API; one-shot Docker local inference servers. Not a fit: production-grade serving (officially self-described as not production-oriented); broad GGUF quant coverage (the llama.cpp root).
中文生态China availability 国产 / 国内可直接用 / 备案状态 · 快照domestic / direct access / CAC filing · snapshot 2026-10-31
- 出品属地Origin
- 国外出品foreign-made Royal Lab（theroyallab）——GitHub 组织名义的开源社区项目。Royal Lab (theroyallab) — an open-source community project under a GitHub org name.
- 国内使用Direct access
- 大陆可直连direct OK GitHub 仓/raw README 直连 200；tabbyapi.com 直连为 Cloudflare 验证页（且按官方警告非官方站点）（2026-10-31）。The GitHub repo and raw README returned 200 direct; tabbyapi.com serves a Cloudflare verification page (and per the official warning is not an official site) (2026-10-31).
- 备案状态CAC filing
- 未见于备案名单not in filing list：未见于网信办公示名单（2026-10-31 检索）；开源项目未面向大陆公众提供服务，备案不适用。not in the CAC public registry (searched 2026-10-31); an open-source project not serving the mainland public, so filing does not apply.
备案信息以网信办《生成式人工智能服务已备案信息》公示名单为准（按批次滚动更新，本站只抄编号不解读）；「大陆可直连」是本站从大陆网络直连实测的结果，不是厂商承诺。GitHub 类站点当日实测有波动，口径写在依据里。Filing status is quoted from the CAC public registry (updated batch by batch; this site copies the number and does not interpret it). "Direct access" is this site's own reachability measurement from a mainland network, not a vendor promise. GitHub-hosted sites fluctuated on the measurement day — the note says so.
固定坐标系Fixed coordinate system
8 个维度dimensions, 与同分区其他对象逐项可比each comparable item by item against others in the same section.
- 模型与开放条件Model & openness
ExLlamaV3 官方 API 后端——README 自述「TabbyAPI is also the official API backend server for ExllamaV3」：跑 GPTQ/GGUF 等 ExLlamaV3 支持的本地权重（exllamav3 后端），不是调外部模型的服务。 模型文件放本机目录挂载进进程（Docker 口径：models 目录挂进容器）。
The official API backend for ExLlamaV3 — the README says "TabbyAPI is also the official API backend server for Exllamav3": it runs local weights supported by ExLlamaV3 (the exllamav3 backend) such as GPTQ/GGUF, not a service calling an external model. Model files live in a local directory mounted into the process (Docker wording: mount the models directory into the container).
- 运行位置Where it runs
FastAPI 本地服务（Python 3.10–3.14）；官方 CUDA 镜像在 GHCR（ghcr.io/theroyallab/tabbyapi）， 标签 latest（CUDA 12.8）/cu13/latest-rocm（AMD RX 7900 系）/latest-extras（含可选 embeddings 栈）。 GitHub theroyallab/tabbyAPI AGPL-3.0 1,502★ 推送 2026-10-30（API 核验）； GitHub releases 为 0——发版走 rolling release（README 自述），变更记录线看 commits。
A FastAPI local server (Python 3.10–3.14); the official CUDA image is on GHCR (ghcr.io/theroyallab/tabbyapi), tags latest (CUDA 12.8)/cu13/latest-rocm (AMD RX 7900 series)/latest-extras (with the optional embeddings stack). GitHub theroyallab/tabbyAPI AGPL-3.0 1,502 stars, pushed 2026-10-30 (API verified); GitHub releases count is 0 — releases follow a rolling-release model (README self-description); the change-record line is commits.
- 本地文件Local files
读写本机文件——模型权重目录、config.yml 配置；API 默认 localhost:5000。
Reads and writes local files — the model-weights directory, config.yml; the API defaults to localhost:5000.
- 关机后的任务Tasks after shutdown
常驻本机进程（FastAPI server）；无常驻云面。README 免责声明自述「hobby project…not meant to run on production servers」——生产负载官方建议另寻方案。
A persistent local process (FastAPI server); no persistent cloud surface. The README disclaimer self-describes it as a "hobby project…not meant to run on production servers" — for production workloads the official advice is to look elsewhere.
- 工具与扩展Tools & extensions
OpenAI 兼容 API 面（chat/completions 等端点）+ 管理面（admin UI/status 页，commits 可见 admin 功能演进）； 不是 agent 工具宿主，是给前端（SillyTavern 类）或客户端供推理 API 的服务端。
An OpenAI-compatible API surface (chat/completions and similar endpoints) + an admin surface (admin UI/status page; admin features visible evolving in commits); not an agent-tool host — it is an inference-API server for frontends (SillyTavern-style) or clients.
- 上下文与记忆Context & memory
无跨会话记忆（推理服务面）——上下文长度随 ExLlamaV3 后端配置（max_context 类参数）变化； 对话状态由前端客户端（SillyTavern 类）携带，服务端不存会话历史。
No cross-session memory (it is an inference-serving surface) — context length follows the ExLlamaV3 backend config (max_context-style parameters); conversation state is carried by the frontend client (SillyTavern-style); the server stores no chat history.
- 权限与限制Permissions & limits
本机运行；可配 API key（config.yml）；权限面=本机文件与 GPU。Docker 口径要 --shm-size 调大 （ExLlamaV3 用 POSIX 共享内存做张量并行/CPU MoE 卸载，默认 64MiB 不够——README 明写）。
Runs locally; an API key can be configured (config.yml); the permission surface = local files and GPU. The Docker wording requires a larger --shm-size (ExLlamaV3 uses POSIX shared memory for tensor parallelism and CPU MoE offload; the 64 MiB default is not enough — stated plainly in the README).
- 适合什么任务Best for
适合：ExLlamaV3 生态（GPTQ 系量化）要 OpenAI 兼容 API 的标准答案；要 Docker 一键起 CUDA/ROCm 推理服务。 不适合：要生产级服务化（官方自述非生产向，vLLM/TGI 那条线）；要 GGUF 全量化档广覆盖（llama.cpp 根）。
Fits: the ExLlamaV3 ecosystem (GPTQ-family quantization) wanting an OpenAI-compatible API — the standard answer; one-shot Docker CUDA/ROCm inference servers. Does not fit: production-grade serving (officially self-described as not production-oriented — that is the vLLM/TGI line); broad GGUF quant coverage (the llama.cpp root).
头号误解Top misconception
- 把 tabbyapi.com 当官网——README 官方警告「The ONLY official source for TabbyAPI information is this repository」，其他声称代表 TabbyAPI 的站点/下载/服务均非官方（tabbyapi.com 直连为 Cloudflare 验证页）Treating tabbyapi.com as the official site — the README carries the official warning "The ONLY official source for TabbyAPI information is this repository"; any other site/download/service claiming to represent TabbyAPI is unaffiliated (tabbyapi.com serves a Cloudflare verification page direct)
- 等 GitHub releases——releases 为 0，它是 rolling release，看 commits 与 WikiWaiting for GitHub releases — releases count is 0; it is a rolling release, watch commits and the Wiki
- 当生产服务器用——README 免责声明明写 hobby project、非生产向Using it as a production server — the README disclaimer plainly says hobby project, not production-oriented
- Docker 忘调 --shm-size——ExLlamaV3 张量并行/MoE 卸载走共享内存，默认 64MiB 会出问题（README 明写保留该 flag）Forgetting --shm-size in Docker — ExLlamaV3 tensor parallelism/MoE offload uses shared memory; the 64 MiB default breaks (the README says to keep the flag)
未知项清单Known unknowns
- 支持的量化格式完整矩阵（GPTQ/GGUF 各档支持面，Wiki 有口径未逐条核验）
- config.yml 全部配置项（API key、管理面开关等，未逐条清点）
- embeddings 栈（latest-extras）的具体能力面（README 提及，未核验）
- The full quantization-format support matrix (per-tier GPTQ/GGUF support; the Wiki has wording, not verified item by item)
- All config.yml options (API key, admin-surface toggles, etc.; not enumerated item by item)
- What the embeddings stack (latest-extras) actually covers (mentioned in the README, not verified)
证据来源Evidence sources
判断可回到以下一手源复核。本站核验日Every judgement can be rechecked against the primary sources below. Verified on 2026-10-31，内容更新日content updated on 2026-10-31。
| 类型Type | 名称Name | 链接Link |
|---|---|---|
| repo | theroyallab/tabbyAPI · 仓库（AGPL-3.0；1,502★，推送 2026-10-30，GitHub API 核验；README 官方唯一性警告+Exllamav3 后端+Docker 口径）theroyallab/tabbyAPI · repository (AGPL-3.0; 1,502 stars, pushed 2026-10-30, GitHub API verified; README official-uniqueness warning + Exllamav3 backend + Docker wording) | https://github.com/theroyallab/tabbyAPI |
| changelog | GitHub commits · tabbyAPI（变更记录线：rolling release，无 releases）GitHub commits · tabbyAPI (change-record line: rolling release, no releases) | https://github.com/theroyallab/tabbyAPI/commits/main |
| docs | TabbyAPI · 官方文档（theroyallab.github.io/tabbyAPI，README 挂链的 API 文档徽章指向）TabbyAPI · official docs (theroyallab.github.io/tabbyAPI, the API-docs badge linked from the README) | https://theroyallab.github.io/tabbyAPI |
实测记录Measured runs
本档未跑吞吐与显存实测；结论只覆盖运行时能力与许可。
This site has not completed measured runs. The test protocol is at protocols/runner/_protocol.md.
相关条目Related entries
- llama.cppllama.cpp对照：llama.cpp 是 GGUF 推理根；TabbyAPI 站 ExLlamaV3 后端（GPTQ 系见长）。Contrast: llama.cpp is the GGUF inference root; TabbyAPI stands on the ExLlamaV3 backend (strong on the GPTQ family).
- OllamaOllama对照：Ollama 是 pull 即跑的产品化封装；TabbyAPI 是给前端供 API 的服务端，配置面更工程化。Contrast: Ollama is a productized pull-and-run wrapper; TabbyAPI is an API server for frontends, with a more engineering-oriented configuration surface.
- text-generation-webuitext-generation-webui / textgen同类：都是本地推理服务+Web 管理面生态；TabbyAPI 无 UI 壳、纯 API 服务端。Same family: both belong to the local-inference-server + web-admin ecosystem; TabbyAPI has no UI shell — it is a pure API server.
- ChatGPTChatGPT边界对照：云端对话入口 vs 本机 ExLlamaV3 推理 API——同为 OpenAI 兼容端点，数据边界与模型归属完全不同。Boundary contrast: a cloud chat entry vs a local ExLlamaV3 inference API — both OpenAI-compatible endpoints, completely different data boundaries and model ownership.
本页由This page is generated from the ai-handbook 数据层生成（CC BY 4.0）。data layer (CC BY 4.0).
方法论与坐标系定义见仓库内For the methodology and coordinate definitions, see METHODOLOGY.md.