跳到主要内容Skip to main content
投机取巧 Speculative Speculation

AI 工具AI Tools · Runners

本地模型运行器Local model runners

本地模型运行器：把开源权重模型跑在自己的机器上——Ollama / LM Studio / llama.cpp / vLLM 这类。档案的 model_access / runtime 维写「格式 × 量化档」口径，具体量化数据在 models 站，两边互链不重复。Local model runners — open-weight models running on your own machine (Ollama / LM Studio / llama.cpp / vLLM class). The model_access / runtime axes are written in "format × quant tier" terms; the per-tier numbers live on the models site, cross-linked rather than duplicated.

21 个对象objects 8 个固定维度fixed dimensions 0 个已完整核验fully verified 最后核验Last checked 2026-11-03
  1. 本站按 10 大类收录 AI 产品。编程、对话、图像、视频、音频、研究、自动化、框架、MCP 工具、本地运行器——每类都有明确的收录标准，写在各大类的目录页里。This site files AI products into 10 categories. Coding, chat, image, video, audio, research, automation, frameworks, MCP tools and local runners — each with explicit admission criteria, written on the category page itself.
  2. 大类看目录，形态看字段。一个产品属于哪一类由收录边界决定；它怎么装进你的工作流（IDE / CLI / 厂商云 / 框架 / MCP…）是另一根独立的轴。Category comes from the shelf, form from a field. Which category a product belongs to is decided by admission boundaries; how it installs into your workflow (IDE / CLI / vendor cloud / framework / MCP…) is a separate axis.
  3. 每条档案都是核验快照。字段逐条挂官方出处与核验日期，本站不实测、不排名，未知就写未知。Every entry is a verification snapshot. Each field carries an official source and a checked date; we run no benchmarks, publish no rankings, and write "unknown" when it is unknown.

全部对象All objects

按统一坐标系排列。点开任一对象可看到 模型与开放条件 / 运行位置 / 本地文件 / 关机后的任务 / 工具与扩展 / 上下文与记忆 / 权限与限制 / 适合什么任务 的完整记录、证据链接与未知项清单。Everything on one coordinate system. Open any object to see its full record across Model & openness / Where it runs / Local files / Tasks after shutdown / Tools & extensions / Context & memory / Permissions & limits / Best for, with source links and a list of unknowns.

ExLlamaV3ExLlamaV3

turboderp-orgturboderp-org

ExLlamaV3 是 ExLlamaV2 的接续线：用 EXL3 量化格式在 NVIDIA GPU 上做低延迟本地推理。 The successor to ExLlamaV2: low-latency local inference on NVI

部分核验Partly verified 详情 →Details →

GPT4AllGPT4All

Nomic AI

GPT4All 是「人人设备上跑本地模型」的桌面工具（Nomic AI，MIT）：CPU 优先、桌面开箱、商用无附加条件。 GPT4All is the founding standard-bearer of 'local models on ev

部分核验Partly verified 详情 →Details →

JanJan

Menlo Research

Jan 是「开源版 ChatGPT 桌面替代」的长期主义者：100% 离线跑本地模型，Apache-2.0 主体加一句署名请Jan is the long-standing open-source 'offline ChatGPT replacem

部分核验Partly verified 详情 →Details →

KoboldCppKoboldCpp

LostRuins（KoboldAI 社区）LostRuins (KoboldAI community)

KoboldCpp 是角色扮演/创意写作圈的本地推理老兵：单文件免安装、GGUF、Kobold 系上下文管理传统。 AGPLKoboldCpp is the veteran local runner of the roleplay/creative

部分核验Partly verified 详情 →Details →

LemonadeLemonade

AMD / Lemonade 社区AMD / Lemonade community

Lemonade 是 AMD 生态的本地模型服务器：在本机跑模型，提供 OpenAI 兼容 API， 支持 llama.cpAn AMD-ecosystem local model server: run models on your machin

部分核验Partly verified 详情 →Details →

llama.cppllama.cpp

ggml-org

llama.cpp 是整个本地推理生态的地基：GGUF 格式由它定义，models 站的量化索引按它的口径建， Ollamallama.cpp is the bedrock of the entire local-inference ecosyst

部分核验Partly verified 详情 →Details →

LlamaEdgeLlamaEdge

LlamaEdge 社区（WasmEdge / Second State 生态）LlamaEdge community (WasmEdge / Second State ecosystem)

LlamaEdge：WasmEdge 运行时上的 GGUF 推理 Wasm 应用集（Apache-2.0，1,663★）——LlamaEdge: GGUF inference Wasm apps on the WasmEdge runtime (A

部分核验Partly verified 详情 →Details →

LM StudioLM Studio

Element Labs

LM Studio 是桌面跑本地模型的主流图形入口：应用内下载模型、双运行时（MLX + llama.cpp）、内置本地服务LM Studio is the mainstream graphical entry for local models o

部分核验Partly verified 详情 →Details →

LocalAILocalAI

Mudler T.（mudler）

LocalAI 是「换 URL 就转本地」的多模态运行时：单二进制 OpenAI 兼容前置，底下接多引擎， 文本/语音/视觉LocalAI is the 'change the URL and go local' multimodal runtim

部分核验Partly verified 详情 →Details →

MLC LLMMLC LLM

MLCEngine（CMU 系，mlc-ai）MLCEngine (CMU lineage, mlc-ai)

MLC LLM：编译式跨平台推理引擎（Apache-2.0，23,284★ 推送 2026-10-30）—— 「UniverMLC LLM: a compile-to-native cross-platform inference engine (

部分核验Partly verified 详情 →Details →

MLX（mlx-lm）MLX（mlx-lm）

Apple

MLX 是 Apple 硅的官方本地推理栈：统一内存架构的原生框架（mlx）+ 官方 LLM 运行线（mlx-lm）， MIMLX is Apple's official local-inference stack for Apple silico

部分核验Partly verified 详情 →Details →

MstyMsty

Msty（CloudStack, LLC）

Msty：本地+云端混合的桌面 LLM 应用（闭源，CloudStack, LLC）—— Studio 桌面应用跑本地 GGMsty: a hybrid local+cloud desktop LLM app (closed-source, Clo

部分核验Partly verified 详情 →Details →

OllamaOllama

Ollama Inc.

Ollama 是「本地跑开源模型」的事实标准入口：一条命令装完，从官方模型库一键拉模型， 本机起一个 OpenAI 兼容 AOllama is the de-facto entry point for running open models loc

部分核验Partly verified 详情 →Details →

OpenVINOOpenVINO

IntelIntel

OpenVINO 是 Intel 的推理运行时与模型服务器：先把模型转成 IR，再在 CPU/GPU/NPU 上跑； OVMIntel's inference runtime and model server: convert the model

部分核验Partly verified 详情 →Details →

SGLangSGLang

LMSYS / sgl-project

SGLang 是服务化推理的「前缀复用」一代：RadixAttention 把多轮/共享前缀的 KV 缓存复用做成主轴， aSGLang is the prefix-reuse generation of serving engines: Radi

部分核验Partly verified 详情 →Details →

TabbyAPITabbyAPI

Royal Lab（theroyallab）

TabbyAPI：ExLlamaV3 的官方 API 后端服务器（AGPL-3.0，1,502★ 推送 2026-10-30TabbyAPI: the official API backend server for ExLlamaV3 (AGPL-

部分核验Partly verified 详情 →Details →

vLLMvLLM

vLLM 项目（PyTorch 基金会）vLLM project (PyTorch Foundation)

vLLM 是开源模型的推理与服务化引擎：PagedAttention 把显存管理做成核心竞争力，OpenAI 兼容 servvLLM is an inference and serving engine for open models: Paged

部分核验Partly verified 详情 →Details →

本站目前是「核验快照」，还没有实测数据This site is currently a verification snapshot — no measured runs yet

下面每一个字段都能回到官方原文核对，但我们还没有在同一批任务上跑过这些工具。Every field below can be traced back to the official source, but we have not yet run these tools on the same batch of tasks. 实测协议已经写好（统一任务、统一验收、记录人工介入与返工次数），任务清单也已就绪，The measurement protocol is written (shared tasks, shared acceptance criteria, recording human interventions and rework counts) and the task list is ready, but it is 尚未执行。not yet run.

所以本站给的是能力边界与证据，不是「哪个更好用」的结论。So what this site gives you is capability boundaries and evidence, not a verdict on which is better to use. 真正的选型请用你自己的输入、预算与验收标准跑一遍。For an actual choice, run them yourself against your own inputs, budget and acceptance criteria.

8 个维度，每格都能回溯8 dimensions, every cell traceable

所有对象在同一坐标系下被描述。All objects are described on the same coordinate system. 点开任一对象，每一格下面都有官方源链接与核验日期——不认同可以自己回去查。Open any object and every cell carries an official source link and a checked date — disagree and verify it yourself.

01

模型与开放条件Model & openness

用谁的模型？能不能换第三方？多模型路由怎么配？Whose model does it use? Can you switch to a third party? How is multi-model routing configured?

02

运行位置Where it runs

本地 / 云端 / 混合？索引在本地还是云端？远程控制不等于云端执行。Local, cloud or hybrid? Is the index local or remote? Remote control is not the same as cloud execution.

03

本地文件Local files

能访问多大范围？未打开的大文件怎么读？符号级能力如何？How wide an area can it reach? How does it read large files you have not opened? How capable is it at the symbol level?

04

关机后的任务Tasks after shutdown

电脑关了还能继续跑吗？这是本地形态最常见的误解来源。Does it keep running once the computer is off? This is the most common misconception about local forms.

05

工具与扩展Tools & extensions

工具链有多宽：MCP · LSP · 插件 · Hooks · 自定义指令。How wide is the toolchain: MCP · LSP · plugins · hooks · custom instructions?

06

上下文与记忆Context & memory

索引算法、上下文窗口、会话恢复与分叉、跨会话记忆边界。Indexing algorithm, context window, session resume and fork, and the boundary of cross-session memory.

07

权限与限制Permissions & limits

沙箱与审批边界、凭据管理、数据是否出境、额度如何计算。Sandbox and approval boundaries, credential handling, whether data leaves your machine, and how quota is charged.

08

适合什么任务Best for

最擅长什么任务？什么情况下别用它？What tasks is it best at? When should you not use it?

全部数据一览All data at a glance

同一张表横向对比。点对象名进详情页看逐格证据。The same table side by side. Click an object name to see per-cell evidence.

对象Object形态Form厂商Vendor适合Best for核验日Checked状态Status
DoDODocker Model RunnerDocker Model Runner 本机应用Local app DockerDocker 适合：已用 Docker 的团队、想把模型分发纳入容器流水线、要本地 OpenAI 兼容端点。 不适合：不想装 Docker、要 GPU 极Fits teams already on Docker who want model distribution inside contai 2026-11-03 partial
ExEXExLlamaV3ExLlamaV3 CLI 形态CLI form turboderp-orgturboderp-org 适合：NVIDIA 单机/多卡、追 EXL3 量化与低延迟推理、愿意自己配 CUDA 环境。 不适合：AMD/CPU、想要开箱 GUI、想要Fits NVIDIA single/multi-GPU users chasing EXL3 quantisation and low-l 2026-11-03 partial
GPGPGPT4AllGPT4All 本机应用Local app Nomic AI 适合：低配机器（CPU 优先）跑小模型、要 MIT 无附加条件的桌面方案。 不适合：要新模型开箱（维护节奏放缓——看仓库最近推送）；要服务化Fits: small models on low-spec/CPU machines, a clean MIT desktop optio 2026-10-30 partial
JaJAJanJan 本机应用Local app Menlo Research 适合：要开源的桌面 GUI 跑离线模型（对比闭源的 LM Studio）。 不适合：要服务化多模型部署；要开箱即用最新模型（模型获取体验弱于Fits: auditable open-source offline desktop chat, built-in OpenAI-comp 2026-10-30 partial
KoKOKoboldCppKoboldCpp 本机应用Local app LostRuins（KoboldAI 社区）LostRuins (KoboldAI community) 适合：角色扮演/长上下文创意写作、要单文件免安装、要 KoboldAI 生态兼容。 不适合：要 OpenAI 兼容主叙事（有但非主轴）；要 Fits: long-context roleplay and creative writing, zero-install single 2026-10-30 partial
LeLELemonadeLemonade 本机应用Local app AMD / Lemonade 社区AMD / Lemonade community 适合：AMD Ryzen AI / NPU 用户、想要本地 OpenAI 兼容端点、离线或隐私场景。 不适合：NVIDIA 独占优化、想要开Fits AMD Ryzen AI / NPU users who want a local OpenAI-compatible endpo 2026-11-03 partial
llLLllama.cppllama.cpp CLI 形态CLI form ggml-org 适合：要细粒度控制（量化档/后端/参数）、消费级硬件跑 GGUF、给下游工具做底座。 不适合：要开箱 GUI（用 LM Studio/KobFits: fine-grained control (quant tiers/backends/params), GGUF on cons 2026-10-30 partial
LlLLLlamaEdgeLlamaEdge CLI 形态CLI form LlamaEdge 社区（WasmEdge / Second State 生态）LlamaEdge community (WasmEdge / Second State ecosystem) 适合：要 WASM 可移植路线——一份 .wasm 应用跨 CPU/GPU 设备；边缘/嵌入式场景（WasmEdge 生态）。 不适合：要活Fits: the portable-WASM route, edge/embedded devices; WasmEdge-ecosyst 2026-10-31 partial
LMLMLM StudioLM Studio 本机应用Local app Element Labs 适合：要图形界面跑本地模型、想在同一应用里试 agent（Bionic）。 不适合：要无头服务器部署（用 llama.cpp/vLLM）；要Fits: local-model users who dislike the CLI, trying agents and voice t 2026-10-30 partial
LoLOLocalAILocalAI CLI 形态CLI form Mudler T.（mudler） 适合：一个二进制换 URL 就把现有 OpenAI 客户端转本地、要文本之外（语音/图像）多模态负载同一运行时。 不适合：只要单一 LLM Fits: minimal-change local switch for existing OpenAI clients, CPU-onl 2026-10-30 partial
MLMLMLC LLMMLC LLM CLI 形态CLI form MLCEngine（CMU 系，mlc-ai）MLCEngine (CMU lineage, mlc-ai) 适合：要「一套引擎多平台部署」——尤其移动端（iOS/Android 原生 Metal/OpenCL）与浏览器（WebLLM） 这类 llaFits: one engine deployed across platforms (especially mobile and brow 2026-10-31 partial
MLMLMLX（mlx-lm）MLX（mlx-lm） CLI 形态CLI form Apple 适合：Apple 硅机器要原生性能（统一内存红利）、要在 Mac 上微调（mlx-lm 带微调线）。 不适合：非 Apple 硬件；要跨平台Fits: native performance on M-series Macs (unified-memory dividend), L 2026-10-30 partial
MsMSMstyMsty 本机应用Local app Msty（CloudStack, LLC） 适合：要「本地+云端一个桌面应用」的混合使用；企业私有 LLM（Nexus 线）。 不适合：要开源可审计栈（闭源）；只要纯 CLI 服务化（Fits: hybrid local+cloud use in one app; private enterprise LLMs (Nexu 2026-10-31 partial
OlOLOllamaOllama CLI 形态CLI form Ollama Inc. 适合：一条命令把开源模型跑起来、要 OpenAI 兼容本地 API、桌面与服务器通吃。 不适合：要极致吞吐/服务化调度（那是 vLLM/SGFits: getting DeepSeek/Qwen/Llama running with one command, OpenAI-com 2026-10-30 partial
OpOPOpenVINOOpenVINO CLI 形态CLI form IntelIntel 适合：Intel CPU/GPU/NPU、边缘部署、要把模型压到本地运行。 不适合：NVIDIA 独占优化场景（TensorRT-LLM/vFits Intel CPU/GPU/NPU, edge deployment and squeezing models onto loca 2026-11-03 partial
SGSGSGLangSGLang CLI 形态CLI form LMSYS / sgl-project 适合：agent 多轮调用/共享 system prompt 的服务化场景（前缀复用收益直接）、要结构化输出语言特性。 不适合：单机开箱跑量Fits: agent multi-turn calls, serving workloads with shared system pro 2026-10-30 partial
TaTATabbyAPITabbyAPI CLI 形态CLI form Royal Lab（theroyallab） 适合：ExLlamaV3 生态（GPTQ 系量化）要 OpenAI 兼容 API 的标准答案；要 Docker 一键起 CUDA/ROCm Fits: the ExLlamaV3 (GPTQ family) wanting an OpenAI-compatible API; on 2026-10-31 partial
NVNVNVIDIA TensorRT-LLMNVIDIA TensorRT-LLM CLI 形态CLI form NVIDIANVIDIA 适合：NVIDIA GPU 集群、要生产级吞吐与低延迟、已有容器/编译流水线。 不适合：单机消费级显卡开箱、AMD/CPU 部署、想要 GGFits NVIDIA GPU clusters that need production throughput and low laten 2026-11-03 partial
teTEtext-generation-webui / textgentext-generation-webui / textgen 本机应用Local app oobabooga 适合：要扩展生态与多 loader 灵活度、社区玩法（角色卡/微调衔接）。 不适合：要开箱极简（界面信息密度高）；要 AGPL 之外的许可做Fits: power users wanting loader/extension flexibility, community play 2026-10-30 partial
TeTEText Generation Inference（TGI）Text Generation Inference（TGI） CLI 形态CLI form Hugging Face 适合：HF 生态内既有 TGI 部署的延续（权重/客户端/文档同一门）、要官方 Docker 线。 不适合：新项目选型——官方公告点名推荐 Fits: HF-Hub-heavy serving deployments, official Docker/client pairing 2026-10-30 partial
vLVLvLLMvLLM CLI 形态CLI form vLLM 项目（PyTorch 基金会）vLLM project (PyTorch Foundation) 适合：多模型/多请求的服务化部署、要吞吐与显存效率、生产环境 OpenAI 兼容端点。 不适合：单机开箱跑量化档（llama.cpp/OllFits: production serving of many models/requests, GPU-cluster throughp 2026-10-30 partial

为什么这里不给总分排名Why there is no overall ranking

这不是省略，是方法上的明确取舍。This is not an omission but a deliberate methodological choice.

  • 缺统一实测，就不给分。跨工具的跑分口径不同——题目、预算、执行环境都不一样，A 的 90 分和 B 的 88 分不可比。No shared measurement, no score. Benchmarks across tools use different rules — different questions, budgets and environments. A 90 and a B 88 are not comparable.
  • 「未知」是合法答案。查不到就写「未知」并说明为什么，不用推测填充。缺信息本身也是信息。"Unknown" is a legitimate answer. If it cannot be found, we write "unknown" and explain why, instead of filling the gap with guesses. Missing information is itself information.
  • 来源链接、类型与核验日都在详情页里，可以逐条回去复核。The source link, its kind and the checked date are all on the detail page, so you can walk any of them back and re-check.
  • lifecycle 单独标注。核验日新不等于数据有效——上游归档后数据会失效，所以单独标 active / maintenance / archived。lifecycle is marked separately. A recent check date does not mean the data is still valid — it goes stale when upstream archives the repo, so active / maintenance / archived are tracked separately.
  • 不同族不混排。找现成的、自己搭的、接上与跑起来的是三种角色，坐标系不同；硬合成一张总表只会制造假的可比性。Families are not mixed. Ready-made tools, things you build yourself, and plug-in-and-run components are three different roles with different coordinate systems; forcing them into one master table would manufacture false comparability.

可信度标记怎么读How to read the confidence marks

详情页与上表都带Both the detail pages and the table above carry confidence，它反映, which reflects 本站数据的完整度how complete this site's data is，不是对产品的评价。, not a judgement of the product.

verified8 个维度均有官方源支撑，且核验日在 90 天内all 8 dimensions have official sources and were checked within 90 days
partial部分维度标为未知，或官方文档不可访问，或核验日超过 90 天some dimensions are marked unknown, or the official docs were unreachable, or checked over 90 days ago
stale官方已发布重大变化，本站尚未核验the vendor has shipped a significant change that this site has not re-checked

本站当前的完整度Current completeness on this site

数据层用Recheck this figure with node scripts/audit-gaps.mjs 可随时复核这个数字。. 下表实时计算（本页档案最后核验日 2026-11-03）：Computed from current data; profiles last checked 2026-11-03:

大类Category已补齐Filled说明Meaning
本地运行器Local runners 106/168 开源项目文档最完整，但版本演进快 —— 核验快照最容易过期Open-source projects document the most, but they release fast — these snapshots go stale first

全站合计 1372/1896 维度已补齐Site-wide 1372/1896 dimensions filled（72%）

剩余未补齐项分三类：官方未公开（索引算法、沙箱实现细节本就不对外说明）、需实测才能确定（大仓库表现、CI 无 TTY 行为）、客观渠道不可达。The remaining gaps fall into three kinds: not disclosed by the vendor (indexing algorithms, sandbox internals are simply not documented), requires measurement (behaviour on very large repositories, CI without a TTY), and unreachable through objective channels.

我们选择留白而不是填「已支持」——错误的成本最终由使用者承担。We chose to leave a blank rather than fill in "supported" — the cost of an error is ultimately borne by whoever uses it.

数据来源与复核方式Sources and how to re-check them

全部数据来自开源仓库All data comes from the open-source repository speculcom/ai-handbook（CC BY 4.0）。 (CC BY 4.0).

每个条目都是纯 Markdown + frontmatter，含维度记录、证据链接、核验日与可信度标记。仓库内含：Every entry is plain Markdown plus frontmatter, carrying dimension records and an evidence chain with links, checked dates and confidence marks. The repository contains:

  • tools/METHODOLOGY.md —— 方法论总纲与核心规则the methodology charter and core rules
  • tools/axes/ —— 每个维度的定义与判定标准（含正反例）the definition and decision criteria for each dimension (with positive and negative examples)
  • tools/SCHEMA.md —— 数据格式规范与构建校验规则the data-format spec and build validation rules
  • tools/protocols/<大类><category>/_protocol.md —— 实测协议(the measurement protocol (尚未执行not yet run，故本站暂无实测结论), so this site has no measured conclusions)

发现错误或有新证据，欢迎提 Issue 或 PR——每条修正都会注明依据与影响范围。Found an error or have new evidence? Open an issue or a PR — each correction notes its basis and blast radius.