← Runners

llama.cppllama.cpp

ggml-org

开源Open source MITMIT GGUFGGUF C++C++ 本地推理Local inference llama-serverllama-server partial lifecycle: active 核验Checked 2026-10-30

llama.cpp 是整个本地推理生态的地基：GGUF 格式由它定义，models 站的量化索引按它的口径建， Ollama/LM Studio/KoboldCpp 都在它上面盖楼。MIT、纯 C/C++、llama-server 提供官方 REST 线。

llama.cpp is the bedrock of the entire local-inference ecosystem: GGUF is defined here, the models-site quant index is built to its yardstick, and Ollama/LM Studio/KoboldCpp all build on top of it. MIT, pure C/C++, with an official REST line via llama-server.

适合与不适合When to use it, when not to

适合：要控制量化档/后端/每个参数的用户、消费级 GPU 与 Apple 硅跑 GGUF、给自研工具做推理底座。不适合：要开箱 GUI；要多模型高并发服务化（那是 vLLM/SGLang 的调度强项）。

Fits: fine-grained control (quant tiers/backends/params), GGUF on consumer GPUs and Apple silicon, powering in-house tools. Not a fit: out-of-the-box GUIs; multi-model high-concurrency serving.

中文生态China availability 国产 / 国内可直接用 / 备案状态 · 快照domestic / direct access / CAC filing · snapshot 2026-10-21

出品属地Origin: 国外出品foreign-made　ggml-org（Georgi Gerganov 主导的开源组织）。ggml-org (open-source org led by Georgi Gerganov).
国内使用Direct access: 大陆可直连direct OK　github.com/ggml-org/llama.cpp 与 llama.app 直连实测 200（2026-10-21）；llama.cpp 顶级域直抓 000（网络受限，非死亡判定）。github.com/ggml-org/llama.cpp and llama.app measured 200 direct (2026-10-21); the llama.cpp apex domain measured 000 (network-limited, not a death verdict).
备案状态CAC filing: 未见于备案名单not in filing list：未见于网信办公示名单（2026-10-21 检索）；开源项目未面向大陆公众提供服务，备案不适用。not in the CAC public registry (searched 2026-10-21); the open-source project does not serve the mainland public, so filing does not apply.

备案信息以网信办《生成式人工智能服务已备案信息》公示名单为准（按批次滚动更新，本站只抄编号不解读）；「大陆可直连」是本站从大陆网络直连实测的结果，不是厂商承诺。GitHub 类站点当日实测有波动，口径写在依据里。Filing status is quoted from the CAC public registry (updated batch by batch; this site copies the number and does not interpret it). "Direct access" is this site's own reachability measurement from a mainland network, not a vendor promise. GitHub-hosted sites fluctuated on the measurement day — the note says so.

固定坐标系Fixed coordinate system

8 个维度dimensions, 与同分区其他对象逐项可比each comparable item by item against others in the same section.

模型与开放条件Model & openness: GGUF 格式的生态根——models 站量化索引就是按 llama.cpp 口径建的；safetensors 可转换；支持模型清单随合入滚动变化，未逐条清点。
The ecosystem root of the GGUF format — the models-site quantization index is built to llama.cpp's framing; safetensors convertible; the supported-model list rolls with merges, not inventoried item by item.
运行位置Where it runs: C/C++ 推理库 + llama-server（REST API）+ 各后端（CUDA/Metal/Vulkan/CPU 等，docs/build.md 口径）；官方安装入口已含 llama.app 脚本（README 直抓所见）。GitHub 132,794★（API 核验 2026-10-21）。
A C/C++ inference library + llama-server (REST API) + various backends (CUDA/Metal/Vulkan/CPU etc., per docs/build.md); the official install entry now includes the llama.app script (as seen in the README, direct-fetched). GitHub 132,794★ (API verified 2026-10-21).
本地文件Local files: 全本地：模型文件、对话输出都在自有环境；未见官方云线——第三方云托管不算官方。
Fully local: model files and conversation outputs live in your own environment; no official cloud line seen — third-party cloud hosting does not count as official.
关机后的任务Tasks after shutdown: llama-server 常驻即服务；并发/量化调度细节未逐条核验。
llama-server stays resident as a service; concurrency/quantization scheduling details not verified item by item.
工具与扩展Tools & extensions: server 侧有 function calling 文档线；MCP 支持状态未核验到官方声明。
The server side has a function-calling docs line; MCP support status — no official statement verified.
上下文与记忆Context & memory: 上下文长度按运行参数配置（-c 等）；KV 缓存量化有官方文档（learn 站 KV 缓存词条 refs 回链先例）。
Context length is set by runtime parameters (-c etc.); KV-cache quantization has official docs (precedent: the learn-site KV-cache entry's refs backlink).
权限与限制Permissions & limits: 本地进程无账号体系；llama-server 鉴权由部署方自配；未见云端权限面。
A local process with no account system; llama-server authentication is configured by the deploying party; no cloud permission surface seen.
适合什么任务Best for: 适合：要细粒度控制（量化档/后端/参数）、消费级硬件跑 GGUF、给下游工具做底座。不适合：要开箱 GUI（用 LM Studio/KoboldCpp）；要多模型高吞吐服务化（vLLM/SGLang）。
Fits: fine-grained control (quantization tiers/backends/parameters), running GGUF on consumer hardware, serving as the base for downstream tools. Does not fit: wanting a GUI out of the box (use LM Studio/KoboldCpp); multi-model high-throughput service deployment (vLLM/SGLang).

头号误解Top misconception

把 GGUF 支持面当「所有模型都有量化档」——量化档由第三方量化者产出，看 models 站索引Treating GGUF support as 'every model has a quant build' — quant builds come from third-party quantizers; check the models-site index
只认命令行——llama-server 是官方 REST 服务线，很多工具在它上面跑Knowing only the CLI — llama-server is the official REST line many tools run on
用旧域名印象找官网——README 现推 llama.app 安装脚本（2026-10-21 直抓所见）Using an old-domain habit — the README now promotes the llama.app install script (direct-fetched 2026-10-21)

未知项清单Known unknowns

支持模型全清单（随合入滚动，未逐条清点）
MCP 支持状态（未核验到官方声明）
llama.app 与 GitHub 仓的官方关系细节（README 直抓所见安装入口，站点归属未另核）

The full supported-model list (rolls with merges, not inventoried item by item)
MCP support status (no official statement verified)
Details of the official relationship between llama.app and the GitHub repo (the install entry was seen in the README direct fetch; site ownership not separately verified)

证据来源Evidence sources

判断可回到以下一手源复核。本站核验日Every judgement can be rechecked against the primary sources below. Verified on 2026-10-30，内容更新日content updated on 2026-10-21。

类型Type	名称Name	链接Link
repo	ggml-org/llama.cpp · 仓库（MIT；132,794★，最近推送 2026-10-30，GitHub API 核验）ggml-org/llama.cpp · repo (MIT; 132,794★, last push 2026-10-30, GitHub API verified)	https://github.com/ggml-org/llama.cpp
changelog	ggml-org/llama.cpp · Releases（changelog，含 nightly）ggml-org/llama.cpp · Releases (changelog, incl. nightly)	https://github.com/ggml-org/llama.cpp/releases
docs	llama.app · 官方安装入口（README 直抓所见）llama.app · official install entry (as seen in the README, direct-fetched)	https://llama.app

实测记录Measured runs

本档未跑吞吐与显存实测；结论只覆盖运行时能力与许可。档案的格式/量化维直接复用该快照出处。运行器专用实测协议本站未立。

This site has not completed measured runs. The models-site guides' serving-deep-dive.md has already verified llama.cpp's official framing (compared against vLLM/SGLang), and this dossier's format/quantization axes directly reuse that snapshot's sources. No runner-specific test protocol exists on this site yet.