vLLMvLLM
English summaries and interface labels are available. Detailed source notes below are currently in Chinese.
vLLM 是开源模型的推理与服务化引擎：PagedAttention 把显存管理做成核心竞争力，OpenAI 兼容 server 直接对接现有客户端。 Apache-2.0、PyTorch 基金会生态。数字口径：官方自报性能数字在 models 站 guides 已核验快照，本站不实测。
vLLM is an inference and serving engine for open models: PagedAttention makes memory management its core strength, and the OpenAI-compatible server plugs straight into existing clients. Apache-2.0, PyTorch Foundation umbrella. Numbers: vendor-claimed performance figures live in the models-site guides snapshot; this site does not benchmark.
适合与不适合When to use it, when not to
适合：生产部署多模型/高并发推理服务、GPU 集群吞吐优先。 不适合：个人单机开箱（Ollama 更省事）、GGUF 量化档深度依赖（llama.cpp 路线）。
Fits: production serving of many models/requests, GPU-cluster throughput, OpenAI-compatible endpoints. Not a fit: single-machine casual use (Ollama is easier); deep GGUF dependence.
中文生态China availability 国产 / 国内可直接用 / 备案状态 · 快照domestic / direct access / CAC filing · snapshot 2026-10-21
- 出品属地Origin
- 国外出品foreign-made vLLM 项目（UC Berkeley 起源，现归 PyTorch 基金会生态）。vLLM project (UC Berkeley origin, now under the PyTorch Foundation umbrella).
- 国内使用Direct access
- 大陆可直连direct OK docs.vllm.ai 直连实测 200（2026-10-21）。docs.vllm.ai measured 200 direct (2026-10-21).
- 备案状态CAC filing
- 未见于备案名单not in filing list：未见于网信办公示名单（2026-10-21 检索）；开源项目未面向大陆公众提供服务，备案不适用。not in the CAC public registry (searched 2026-10-21); the open-source project does not serve the mainland public, so filing does not apply.
备案信息以网信办《生成式人工智能服务已备案信息》公示名单为准（按批次滚动更新，本站只抄编号不解读）；「大陆可直连」是本站从大陆网络直连实测的结果，不是厂商承诺。GitHub 类站点当日实测有波动，口径写在依据里。Filing status is quoted from the CAC public registry (updated batch by batch; this site copies the number and does not interpret it). "Direct access" is this site's own reachability measurement from a mainland network, not a vendor promise. GitHub-hosted sites fluctuated on the measurement day — the note says so.
固定坐标系Fixed coordinate system
8 个维度dimensions, 与同分区其他对象逐项可比each comparable item by item against others in the same section.
- 模型与开放条件Model & openness
safetensors/HF 系权重为主（GGUF 支持有限——models 站口径：量化服务化首选仍是 llama.cpp 路线）； 支持模型清单以官方 docs 为准，未逐条清点。
safetensors/HF-line weights are the mainstay (GGUF support is limited — models-site framing: the first choice for quantized serving remains the llama.cpp route); the supported-model list follows the official docs, not inventoried item by item.
- 运行位置Where it runs
服务化推理引擎：PagedAttention 显存管理是官方招牌（自报数字在 models 站 guides serving-deep-dive.md 已核验快照，出处同源）；GitHub 95,878★（API 核验 2026-10-21）。
A serving inference engine: PagedAttention memory management is the official signature (the self-reported figures were verified as a snapshot in the models-site guides serving-deep-dive.md, same source); GitHub 95,878★ (API verified 2026-10-21).
- 本地文件Local files
自托管部署，数据在自有环境；无官方托管云线（第三方云托管不算官方）。
Self-hosted deployment, data in your own environment; no official managed cloud line (third-party cloud hosting does not count as official).
- 关机后的任务Tasks after shutdown
产品形态就是常驻服务（OpenAI 兼容 server）；并发/批处理调度是主轴。
The product form is a resident service (an OpenAI-compatible server); concurrency/batch scheduling is the main axis.
- 工具与扩展Tools & extensions
OpenAI 兼容 API 含 tool calling；MCP 支持状态未逐条核验。
The OpenAI-compatible API includes tool calling; MCP support status not verified item by item.
- 上下文与记忆Context & memory
长上下文按模型与显存配置；前缀缓存等优化在官方文档线，细节未逐条核验。
Long context depends on model and GPU-memory configuration; optimizations like prefix caching are in the official docs line, details not verified item by item.
- 权限与限制Permissions & limits
自托管无账号体系；API 鉴权由部署方自配；多租户隔离与配额面未逐条核验。
Self-hosted with no account system; API authentication configured by the deploying party; multi-tenant isolation and quota surfaces not verified item by item.
- 适合什么任务Best for
适合：多模型/多请求的服务化部署、要吞吐与显存效率、生产环境 OpenAI 兼容端点。 不适合：单机开箱跑量化档（llama.cpp/Ollama 更省事）；GGUF 深度依赖场景。
Fits: multi-model/multi-request serving deployment, wanting throughput and GPU-memory efficiency, production OpenAI-compatible endpoints. Does not fit: single-machine out-of-the-box quantized tiers (llama.cpp/Ollama are less hassle); scenarios deeply dependent on GGUF.
头号误解Top misconception
- 把「跑开源模型」都当等价——vLLM 强在服务化调度，单机单模型开箱它不是最省事Treating 'runs open models' as equivalent everywhere — vLLM's strength is serving scheduling, not casual single-model use
- 把 PagedAttention 自报数字当本站实测——那是官方口径快照，本站不实测Taking PagedAttention vendor numbers as our own benchmarks — they are an official-claim snapshot; this site does not benchmark
- 指望 GGUF 一等公民支持——量化服务化首选 llama.cpp 路线（models 站口径）Expecting first-class GGUF support — quantized serving prefers the llama.cpp line (models-site yardstick)
未知项清单Known unknowns
- 支持模型全清单（未逐条清点）
- MCP 支持状态（未逐条核验）
- 前缀缓存/chunked prefill 等优化的当前默认开关（未逐条核验）
- The full supported-model list (not inventoried item by item)
- MCP support status (not verified item by item)
- The current default switches of optimizations like prefix caching/chunked prefill (not verified item by item)
证据来源Evidence sources
判断可回到以下一手源复核。本站核验日Every judgement can be rechecked against the primary sources below. Verified on 2026-10-30，内容更新日content updated on 2026-10-21。
| 类型Type | 名称Name | 链接Link |
|---|---|---|
| repo | vllm-project/vllm · 仓库（Apache-2.0；95,878★，最近推送 2026-10-30，GitHub API 核验）vllm-project/vllm · repo (Apache-2.0; 95,878★, last push 2026-10-30, GitHub API verified) | https://github.com/vllm-project/vllm |
| docs | vLLM · 官方文档vLLM · official docs | https://docs.vllm.ai |
| changelog | vllm-project/vllm · Releases（changelog）vllm-project/vllm · Releases (changelog) | https://github.com/vllm-project/vllm/releases |
实测记录Measured runs
本档未跑吞吐与显存实测；结论只覆盖运行时能力与许可。 本档案复用同一快照出处，不重复抓取。运行器专用实测协议本站未立。
This site has not completed measured runs. The models-site guides serving-deep-dive.md has already verified vLLM's official framing (PagedAttention self-reported figures); this dossier reuses the same snapshot source without re-fetching. No runner-specific test protocol exists on this site yet.
第一次怎么用
1. 在 NVIDIA GPU 机器上安装 vLLM，先跑 vllm --help 确认版本。 2. 用 vllm serve <model> 起 OpenAI 兼容服务；模型名与上下文长度按官方文档配。 3. 多人并发前先测显存峰值与 tokens/s，再调 batch 与并行参数。 4. PagedAttention 管显存，但上下文越长 KV 缓存越吃显存。 5. 生产环境加监控与限流；不要直接把默认参数暴露到公网。
pip install vllm
vllm serve <model> --host 0.0.0.0 --port 8000
# 然后按 OpenAI 兼容客户端调用；参数以官方文档为准相关条目Related entries
- SGLangSGLang同为服务化推理引擎：SGLang 主打 RadixAttention 与 agent 工作流前缀复用。Also a serving inference engine: SGLang leads on RadixAttention and prefix reuse for agent workflows.
- llama.cppllama.cpp单机 GGUF 路线的根；两条路线在 models 站 guides 有对照。The root of the single-machine GGUF route; the two routes are compared in the models-site guides.
- OpenAI Agents SDKOpenAI Agents SDK另一层 · 框架与运行时Other layer · Frameworks & runtimes
- LlamaIndex FrameworkLlamaIndex Framework另一层 · 框架与运行时Other layer · Frameworks & runtimes
本页由This page is generated from the ai-handbook 数据层生成（CC BY 4.0）。data layer (CC BY 4.0).
方法论与坐标系定义见仓库内For the methodology and coordinate definitions, see METHODOLOGY.md.