MLC LLMMLC LLM
MLC LLM：编译式跨平台推理引擎（Apache-2.0，23,284★ 推送 2026-10-30）—— 「Universal LLM Deployment Engine with ML Compilation」：把模型编译成目标平台原生码， 一套 MLCEngine 覆盖 NVIDIA/AMD/Intel GPU、Apple Metal（含 iOS）、浏览器 WebGPU/WASM（WebLLM）、 Android OpenCL，全部走 OpenAI 兼容 API。两条根口径：safetensors 编译路线 ≠ GGUF 预量化路线； 发版线在 PyPI 不在 GitHub releases（名义 v0.20.0/dev 标签 v0.26.dev0）。
MLC LLM: a compile-to-native cross-platform inference engine (Apache-2.0, 23,284 stars, pushed 2026-10-30) — "Universal LLM Deployment Engine with ML Compilation": compile models into platform-native code, one MLCEngine covering NVIDIA/AMD/Intel GPUs, Apple Metal (incl. iOS), browser WebGPU/WASM (WebLLM) and Android OpenCL, all behind an OpenAI-compatible API. Two root wordings: the safetensors-compile route ≠ the GGUF pre-quantization route; the release line is on PyPI, not GitHub releases (nominally v0.20.0 / dev tag v0.26.dev0).
适合与不适合When to use it, when not to
适合：一套引擎多平台部署（尤其移动端与浏览器端）；编译期优化换性能的工程路线。 不适合：GGUF 即下即跑的省事路线（llama.cpp/Ollama）；不想碰编译步骤的用户。
Fits: one engine deployed across platforms (especially mobile and browser ends); an engineering route trading compile-time optimization for performance. Not a fit: the GGUF download-and-run convenience route (llama.cpp/Ollama); users who do not want to touch a compile step.
中文生态China availability 国产 / 国内可直接用 / 备案状态 · 快照domestic / direct access / CAC filing · snapshot 2026-10-31
- 出品属地Origin
- 国外出品foreign-made MLCEngine / mlc-ai（CMU 系开源社区，LMSYS/MLC 生态）。MLCEngine / mlc-ai (CMU-lineage open-source community, LMSYS/MLC ecosystem).
- 国内使用Direct access
- 直连不可达direct blocked llm.mlc.ai 直连 000（网络受限，非死链）；GitHub 仓与 raw README 直连 200（2026-10-31）。llm.mlc.ai measured 000 direct (network restricted, not dead); the GitHub repo and raw README returned 200 direct (2026-10-31).
- 备案状态CAC filing
- 未见于备案名单not in filing list：未见于网信办公示名单（2026-10-31 检索）；开源项目未面向大陆公众提供服务，备案不适用。not in the CAC public registry (searched 2026-10-31); an open-source project not serving the mainland public, so filing does not apply.
备案信息以网信办《生成式人工智能服务已备案信息》公示名单为准（按批次滚动更新，本站只抄编号不解读）；「大陆可直连」是本站从大陆网络直连实测的结果，不是厂商承诺。GitHub 类站点当日实测有波动，口径写在依据里。Filing status is quoted from the CAC public registry (updated batch by batch; this site copies the number and does not interpret it). "Direct access" is this site's own reachability measurement from a mainland network, not a vendor promise. GitHub-hosted sites fluctuated on the measurement day — the note says so.
固定坐标系Fixed coordinate system
8 个维度dimensions, 与同分区其他对象逐项可比each comparable item by item against others in the same section.
- 模型与开放条件Model & openness
自带编译式推理引擎（MLCEngine）——不是调外部模型的服务，是把模型编译成目标平台的原生执行码 （「Universal LLM Deployment Engine with ML Compilation」官方定位）。模型侧吃 safetensors 系 （HF 权重编译路线），与 llama.cpp 的 GGUF 预量化路线是两条根。
Ships its own compiled inference engine (MLCEngine) — not a service calling an external model; it compiles models into native executables for the target platform (the official positioning: "Universal LLM Deployment Engine with ML Compilation"). On the model side it takes the safetensors family (the HF-weights-compile route), a different root from llama.cpp's pre-quantized GGUF route.
- 运行位置Where it runs
编译-部署型 CLI/库（pip mlc-llm / mlc_llm 命令）；覆盖平台矩阵是它的主打面： NVIDIA/AMD/Intel GPU（CUDA/Vulkan/ROCm/OpenCL）、Apple Metal（含 iOS/iPadOS）、 Web 浏览器（WebGPU+WASM，姊妹项目 WebLLM）、Android（OpenCL）——README 平台表逐格标注。 GitHub mlc-ai/mlc-llm Apache-2.0 23,284★ 推送 2026-10-30（API 核验）； GitHub releases 名义停 v0.20.0（dev 标签 v0.26.dev0）——发版线在 PyPI（mlc-llm 包）。
A compile-and-deploy CLI/library (pip mlc-llm / the mlc_llm command); the platform matrix is its headline surface: NVIDIA/AMD/Intel GPUs (CUDA/Vulkan/ROCm/OpenCL), Apple Metal (incl. iOS/iPadOS), web browsers (WebGPU+WASM, the sister project WebLLM), Android (OpenCL) — the README platform table marks cell by cell. GitHub mlc-ai/mlc-llm Apache-2.0 23,284 stars, pushed 2026-10-30 (API verified); GitHub releases nominally stop at v0.20.0 (dev tag v0.26.dev0) — the release line is on PyPI (the mlc-llm package).
- 本地文件Local files
读写本机文件——模型权重（编译产物缓存在本地）、编译配置 JSON；REST server 跑在本机端口。 编译步骤本身产出平台特定产物（这是它与「下载即跑」型 runner 的操作差）。
Reads and writes local files — model weights (compiled artifacts cached locally), compile-config JSON; the REST server runs on a local port. The compile step itself produces platform-specific artifacts (its operational difference from "download-and-run" runners).
- 关机后的任务Tasks after shutdown
REST server 是常驻本机进程（OpenAI 兼容 API）；无常驻云面——纯本地/自托管。
The REST server is a persistent local process (OpenAI-compatible API); no persistent cloud surface — purely local/self-hosted.
- 工具与扩展Tools & extensions
不是 agent 工具面——它的「工具」是命令行与 API：mlc_llm generate/generate-config/compile 子命令 + MLCEngine 的 OpenAI 兼容 REST API（REST server/python/javascript/iOS/Android 同一引擎同一编译器背书——README 口径）。
Not an agent-tool surface — its "tools" are the command line and the API: the mlc_llm generate/generate-config/compile subcommands + MLCEngine's OpenAI-compatible REST API (REST server/python/javascript/iOS/Android backed by the same engine and same compiler — README wording).
- 上下文与记忆Context & memory
无跨会话记忆（推理服务面）——上下文长度按编译时模型配置与运行时参数决定； 对话状态由客户端携带，服务端不存会话。
No cross-session memory (it is an inference-serving surface) — context length follows the compile-time model config and runtime parameters; conversation state is carried by the client, the server stores no sessions.
- 权限与限制Permissions & limits
本机运行无鉴权默认（REST server 本地端口）；权限面=本机文件与 GPU 访问。无云账户、无遥测声明面。
Local runs are unauthenticated by default (REST server on a local port); the permission surface = local files and GPU access. No cloud account, no telemetry disclosure surface.
- 适合什么任务Best for
适合：要「一套引擎多平台部署」——尤其移动端（iOS/Android 原生 Metal/OpenCL）与浏览器（WebLLM） 这类 llama.cpp/Ollama 不覆盖的端；要编译期优化换运行性能的工程路线。 不适合：要 GGUF 即下即跑的省事路线（llama.cpp/Ollama）；不想碰编译步骤的用户。
Fits: "one engine, multi-platform deployment" — especially mobile (iOS/Android native Metal/OpenCL) and browser (WebLLM) ends that llama.cpp/Ollama do not cover; an engineering route that trades compile-time optimization for runtime performance. Does not fit: the convenience route of GGUF download-and-run (llama.cpp/Ollama); users who do not want to touch a compile step.
头号误解Top misconception
- 把 GitHub releases 当发版线——releases 名义停 v0.20.0（标签有 v0.26.dev0），实际发版在 PyPI；看 PyPITreating GitHub releases as the release line — releases nominally stop at v0.20.0 (tag v0.26.dev0); actual releases happen on PyPI — watch PyPI
- 把它当 GGUF 生态——MLC 走 safetensors 编译路线，GGUF 预量化是 llama.cpp 根；模型准备步骤不同Treating it as part of the GGUF ecosystem — MLC follows the safetensors compile route; pre-quantized GGUF is the llama.cpp root; the model-preparation step differs
- 把 llm.mlc.ai 直连 000 当死站——网络受限非停运（GitHub 仓与文档线经代理可达）Reading llm.mlc.ai's 000 direct measurement as a dead site — network restriction, not retirement (the GitHub repo and docs are reachable via proxy)
- 忽略编译步骤——「编译-部署」是它的操作模型，第一次跑要先编译/取编译产物，不是 pull 即跑Ignoring the compile step — "compile then deploy" is its operational model: the first run compiles or fetches compiled artifacts, it is not pull-and-run
未知项清单Known unknowns
- PyPI 当前版本号与上传日期（pypi.org 直连 000 未达，按快照纪律不记未核验数字）
- 支持模型清单的完整面（文档有列表，未逐条清点）
- 编译产物在各端的分发方式（预编译 vs 用户自编译的边界，文档口径未逐条核验）
- The current PyPI version number and upload date (pypi.org measured 000 direct and unreachable; per snapshot discipline no unverified numbers are recorded)
- The complete supported-model list (the docs have a list; not enumerated item by item)
- How compiled artifacts are distributed per platform (precompiled vs user-compiled boundary; the docs wording not verified item by item)
证据来源Evidence sources
判断可回到以下一手源复核。本站核验日Every judgement can be rechecked against the primary sources below. Verified on 2026-10-31，内容更新日content updated on 2026-10-31。
| 类型Type | 名称Name | 链接Link |
|---|---|---|
| repo | mlc-ai/mlc-llm · 仓库（Apache-2.0；23,284★，推送 2026-10-30，GitHub API 核验；README 平台矩阵+MLCEngine 口径）mlc-ai/mlc-llm · repository (Apache-2.0; 23,284 stars, pushed 2026-10-30, GitHub API verified; README platform matrix + MLCEngine positioning) | https://github.com/mlc-ai/mlc-llm |
| changelog | GitHub releases/tags · mlc-llm（变更记录线：releases 名义 v0.20.0、标签 v0.26.dev0——发版线在 PyPI）GitHub releases/tags · mlc-llm (change-record line: releases nominally v0.20.0, tag v0.26.dev0 — the release line is on PyPI) | https://github.com/mlc-ai/mlc-llm/releases |
| docs | MLC LLM · 官方文档站（安装/quick start/引擎口径；llm.mlc.ai 直连 000 网络受限，经代理 200）MLC LLM · official docs site (install/quick start/engine positioning; llm.mlc.ai measured 000 direct due to network restriction, 200 via proxy) | https://llm.mlc.ai/docs/ |
实测记录Measured runs
本档未跑吞吐与显存实测；结论只覆盖运行时能力与许可。
This site has not completed measured runs. The test protocol is at protocols/runner/_protocol.md.
相关条目Related entries
- OllamaOllama对照：Ollama 是 GGUF pull 即跑的产品化封装；MLC 是编译-部署引擎，端覆盖（手机/浏览器）是 Ollama 没有的。Contrast: Ollama is a productized GGUF pull-and-run wrapper; MLC is a compile-and-deploy engine, and its end coverage (phones/browsers) is what Ollama lacks.
- llama.cppllama.cpp两条根对照：llama.cpp 预量化 GGUF 直接跑；MLC 编译 safetensors 出平台原生码。Two roots contrasted: llama.cpp runs pre-quantized GGUF directly; MLC compiles safetensors into platform-native code.
- ChatGPTChatGPT边界对照：云端对话入口（数据在对方侧）vs 本机编译部署引擎（数据不出本机）——OpenAI 兼容 API 让两者对客户端可互换，数据边界完全不同。Boundary contrast: a cloud chat entry (data on their side) vs a local compile-and-deploy engine (data never leaves the machine) — the OpenAI-compatible API makes them interchangeable for clients, while the data boundaries are completely different.
本页由This page is generated from the ai-handbook 数据层生成（CC BY 4.0）。data layer (CC BY 4.0).
方法论与坐标系定义见仓库内For the methodology and coordinate definitions, see METHODOLOGY.md.