← Runners

Text Generation Inference（TGI）Text Generation Inference（TGI）

Hugging Face

开源Open source Apache2.0Apache2.0 服务化推理Inference serving HuggingFace生态Hugging Face ecosystem partial lifecycle: archived 核验Checked 2026-10-30

TGI 是 Hugging Face 官方的服务化推理服务器：权重、客户端、文档同一门生态，Apache-2.0。诚实标注：仓库已于 2026-03-21 归档只读（GitHub API 核验 2026-10-30）；官方 README 公告自陈进入 maintenance mode——只收小修小补，并点名推荐改用 vLLM、SGLang 与 llama.cpp/MLX。存量部署可用，新项目按官方推荐选型。

TGI is Hugging Face's official serving-inference server: weights, clients and docs in one ecosystem, Apache-2.0. Honest note: last push 2026-03-21 (API snapshot) — current serving heat is with vLLM/SGLang; check the cadence before choosing.

适合与不适合When to use it, when not to

适合：HF Hub 重度用户的既有 TGI 部署延续、要官方 Docker/客户端配套。不适合：新项目默认选型——官方公告点名推荐 vLLM/SGLang（服务化）与 llama.cpp/MLX（本地）；单机开箱走 Ollama。

Fits: HF-Hub-heavy serving deployments, official Docker/client pairing. Not a fit: casual single-machine use; default new-project serving choice (compare vLLM/SGLang cadence first).

中文生态China availability 国产 / 国内可直接用 / 备案状态 · 快照domestic / direct access / CAC filing · snapshot 2026-10-21

出品属地Origin: 国外出品foreign-made　Hugging Face（美国）。Hugging Face (US).
国内使用Direct access: 直连不可达direct blocked　huggingface.co 直连实测 000（网络受限，非死亡判定；经代理可达 200）。huggingface.co measured 000 direct (network-limited, not a death verdict; 200 via proxy).
备案状态CAC filing: 未见于备案名单not in filing list：未见于网信办公示名单（2026-10-21 检索）；开源项目未面向大陆公众提供服务，备案不适用。not in the CAC public registry (searched 2026-10-21); the open-source project does not serve the mainland public, so filing does not apply.

备案信息以网信办《生成式人工智能服务已备案信息》公示名单为准（按批次滚动更新，本站只抄编号不解读）；「大陆可直连」是本站从大陆网络直连实测的结果，不是厂商承诺。GitHub 类站点当日实测有波动，口径写在依据里。Filing status is quoted from the CAC public registry (updated batch by batch; this site copies the number and does not interpret it). "Direct access" is this site's own reachability measurement from a mainland network, not a vendor promise. GitHub-hosted sites fluctuated on the measurement day — the note says so.

固定坐标系Fixed coordinate system

8 个维度dimensions, 与同分区其他对象逐项可比each comparable item by item against others in the same section.

模型与开放条件Model & openness: HF Hub 权重原生对接（Hugging Face 自家产品线）；safetensors 为主；支持模型/架构矩阵以官方 docs 为准，未逐条清点。
Native hookup to HF Hub weights (Hugging Face's own product line); safetensors-first; the supported model/architecture matrix follows the official docs, not inventoried item by item.
运行位置Where it runs: 服务化推理服务器（Rust+Python 混合线）；GitHub 10,880★，最近推送 2026-03-21；仓库已于 2026-03-21 由官方归档为只读（GitHub API 核验 2026-10-30：archived=true，页面横幅「archived by the owner on Mar 21, 2026. It is now read-only」）。
A serving inference server (the Rust+Python hybrid line); GitHub 10,880★, last push 2026-03-21; the repository was archived read-only by the owner on 2026-03-21 (GitHub API verified 2026-10-30: archived=true, page banner "archived by the owner on Mar 21, 2026. It is now read-only").
本地文件Local files: 自托管部署，数据在自有环境；HF Hub 拉取权重走网络，推理数据不出部署环境。
Self-hosted deployment, data in your own environment; pulling weights from HF Hub goes over the network, inference data stays inside the deployment environment.
关机后的任务Tasks after shutdown: 产品形态就是常驻服务；连续批处理与流式输出是官方文档主轴，队列细节未逐条核验。
The product form is a resident service; continuous batching and streaming output are the official docs' main axes, queue details not verified item by item.
工具与扩展Tools & extensions: text-generation 客户端线含工具调用；MCP 支持未逐条核验。
The text-generation client line includes tool calling; MCP not verified item by item.
上下文与记忆Context & memory: 上下文长度按模型与部署配置设定；前缀缓存、chunked prefill 等优化的当前默认开关未逐条核验。
Context length is set by model and deployment configuration; the current default switches of optimizations like prefix caching and chunked prefill not verified item by item.
权限与限制Permissions & limits: 自托管无账号体系；API 鉴权由部署方自配；与 HF Hub 交互的 token 权限面未逐条核验。
Self-hosted with no account system; API authentication configured by the deploying party; the token permission surface of HF Hub interactions not verified item by item.
适合什么任务Best for: 适合：HF 生态内既有 TGI 部署的延续（权重/客户端/文档同一门）、要官方 Docker 线。不适合：新项目选型——官方公告点名推荐 vLLM/SGLang 与 llama.cpp/MLX；仓库已归档只读。
Fits: continuing existing TGI deployments inside the HF ecosystem (weights/clients/docs through one door), wanting the official Docker line. Does not fit: new-project selection — the official notice names vLLM/SGLang and llama.cpp/MLX as the recommended alternatives; the repository is archived read-only.

头号误解Top misconception

把 HF 出品当「默认推荐」——官方 README 公告已明确推荐改用 vLLM/SGLang（及 llama.cpp/MLX 这类互操作本地引擎）Assuming 'made by HF' means 'default recommendation' — on the serving track vLLM/SGLang currently carry more heat; check repo cadence
仓库已归档只读（2026-03-21）——不再有安全补丁与新模型支持，存量部署选型要评估维护风险Last push 2026-03-21 (API snapshot) — verify current maintenance before choosing
把 GGUF 当一等公民——TGI 主路线是 safetensorsTreating GGUF as first-class — TGI's main line is safetensors

未知项清单Known unknowns

支持架构矩阵现状（未逐条清点）
归档后是否仍有安全补丁渠道（未见官方说明——仓库只读，按无补丁评估）
MCP 支持状态（未核验到官方声明）

The current state of the supported-architecture matrix (not inventoried item by item)
Whether any security-patch channel remains after archiving (no official statement — the repo is read-only, so plan for none)
MCP support status (no official statement verified)

证据来源Evidence sources

判断可回到以下一手源复核。本站核验日Every judgement can be rechecked against the primary sources below. Verified on 2026-10-30，内容更新日content updated on 2026-10-30。

类型Type	名称Name	链接Link
repo	huggingface/text-generation-inference · 仓库（Apache-2.0；10,880★，最近推送 2026-03-21；2026-03-21 官方归档只读，GitHub API 核验 2026-10-30 archived=true）huggingface/text-generation-inference · repo (Apache-2.0; 10,880★, last push 2026-03-21; archived read-only on 2026-03-21, GitHub API verified 2026-10-30 archived=true)	https://github.com/huggingface/text-generation-inference
docs	TGI · 官方文档TGI · official docs	https://huggingface.co/docs/text-generation-inference/index
changelog	huggingface/text-generation-inference · Releases（changelog）huggingface/text-generation-inference · Releases (changelog)	https://github.com/huggingface/text-generation-inference/releases

实测记录Measured runs

本档未跑吞吐与显存实测；结论只覆盖运行时能力与许可。运行器专用实测协议本站未立。

This site has not completed measured runs. The models-site guides serving-deep-dive.md has covered the serving-inference comparison framing; no runner-specific test protocol exists on this site yet.