← Safety

InspectInspect

UK AI Security Institute

开源Open source MITMIT PythonPython evaleval 沙箱Sandbox MCPMCP 多模型Multiple models 自托管Self-hosted partial lifecycle: active 核验Checked 2026-11-06

核验标记表示资料完整度，不是产品评分。下列未确认项可能影响选型；请按对应来源逐项确认。The confidence mark describes evidence completeness, not a product score. Unconfirmed items below may affect your choice; check them against the relevant sources.

未确认的信息Information not confirmed

全模型 provider 清单与逐家能力差异（官方称 20 余家，未逐条清点）
默认日志目录名、日志文件保留与清理规则
遥测与数据上传条款（含是否有默认上报）：当日未取到官方正文
关机或断网后任务的续跑语义；是否存在托管执行面
Compaction 的默认策略与阈值、Caching 的命中范围
工具审批（Approvers）的默认严格度与可配置档位
CHANGELOG 与 releases 的逐条内容（本批次未验读）
VS Code 扩展与 Inspect View 的能力边界（官方文档导航有 Log Viewer 页，本批次未取正文）

The full provider list and per-provider capability differences (the official claim is 20+; not inventoried)
Default log directory name, retention and cleanup rules
Telemetry and data-upload terms, including any default reporting: no official text captured on the day
Resume semantics after shutdown or network loss, and whether any hosted execution surface exists
Default compaction policy and thresholds; cache hit scope
Default strictness and configurable levels of Tool Approval / Approvers
Changelog and release contents (not reviewed item by item in this pass)
Capability boundaries of the VS Code extension and Inspect View (Log Viewer page text not captured)

Inspect 是英国 AI 安全研究所（与 Meridian Labs）做的开源评测框架：用 Python 把 Dataset、Solver、Scorer 拼成一个 Task，用 inspect eval 在你自己的环境里跑，靠沙箱、工具审批与本地日志把「不受信的模型代码」这件事管起来——官方文档首页同时把它定位成 frontier AI evaluations 的写作与复用工件，而不是托管打分服务。

适合与不适合When to use it, when not to

适合：要自己写评测与 agent 任务（含多 agent 与外部 agent）、要把模型生成的不受信代码放进 Docker / Kubernetes 一类沙箱、要在终端批量跑评测集并在本地日志里逐样本复核、要在多家 provider 之间复用同一份任务定义。不适合：只想在网页控制台点几下就拿到结论、只需要几个打分指标不想维护 Python 工程、把别人跑出的分数当交付物（本站也不这样用）。

中文生态China availability 国产 / 国内可直接用 / 备案状态 · 快照domestic / direct access / CAC filing · snapshot 2026-11-06

出品属地Origin: 国外出品foreign-made　英国 AI 安全研究所（AI Security Institute）与 Meridian Labs 的开源项目；本批次未取到面向中国大陆的服务条款页。An open-source project of the UK AI Security Institute and Meridian Labs; no mainland-facing terms page was captured in this pass.
国内使用Direct access: 直连未核验direct unknown　官方文档站本批次经 HTTP 200 取到正文，但未做大陆网络环境下的直连可用实测，故记未知。The official docs returned HTTP 200 in this pass, but no mainland direct-connection availability measurement was made, so it is recorded as unknown.
备案状态CAC filing: 备案未核验filing unknown：无官方依据，本批次未检索公示，不下「未备案」结论。No official basis; the public registry was not searched in this pass, so no "not filed" conclusion is drawn.

备案信息以网信办《生成式人工智能服务已备案信息》公示名单为准（按批次滚动更新，本站只抄编号不解读）；「大陆可直连」是本站从大陆网络直连实测的结果，不是厂商承诺。GitHub 类站点当日实测有波动，口径写在依据里。Filing status is quoted from the CAC public registry (updated batch by batch; this site copies the number and does not interpret it). "Direct access" is this site's own reachability measurement from a mainland network, not a vendor promise. GitHub-hosted sites fluctuated on the measurement day — the note says so.

固定坐标系Fixed coordinate system

8 个维度dimensions, 与同分区其他对象逐项可比each comparable item by item against others in the same section.

模型与开放条件Model & openness: 官方文档首页：「Built-in support for over 20 model providers」，并写明本地推理支持 HuggingFace、vLLM、SGLang。快速上手示例逐个给 pip install openai + export OPENAI_API_KEY、Anthropic、Google、hf/... + HF_TOKEN 四组，再以 --model openai/... 一类形式指定。全 provider 清单与逐家能力差异：未逐条清点。
The official docs front page states "Built-in support for over 20 model providers" and names local inference with HuggingFace, vLLM and SGLang. The quick-start tabs pair pip install openai + export OPENAI_API_KEY, then Anthropic, Google and hf/... + HF_TOKEN, each run with --model openai/... style selectors. The full provider list and per-provider capability differences were not inventoried.
运行位置Where it runs: pip install inspect-ai 装进自己的 Python 环境，主入口是终端命令 inspect eval（文档首页示例 inspect eval simpleqa.py --model ...）；官方文档首页另列网页版 Inspect View 做监控与可视化、VS Code 扩展辅助编写与调试。执行发生在你的机器与你选的沙箱里；官方托管云线：官方未说明。
pip install inspect-ai into your own Python environment; the main entry point is the terminal command inspect eval (docs example: inspect eval simpleqa.py --model ...). The same page lists the web-based Inspect View tool for monitoring and visualizing evaluations and a VS Code extension for authoring and debugging. Execution happens on your machine inside the sandbox you choose; an official hosted cloud line: not stated officially.
本地文件Local files: 评测是写进 .py 的 Task（Dataset + Solver + Scorer），数据集可直接用 hf_dataset() 从 HuggingFace 载入；官方文档首页把「over 200 pre-built evaluations ready to run on any model」列为自带内容，文档目录另有 Log Files / Inspect View / Task Views 条目。默认日志目录名与文件归置规则本批次未逐字核验。
An evaluation is a Task written in .py (Dataset + Solver + Scorer), and datasets can be loaded straight from HuggingFace with hf_dataset(). The docs front page lists "over 200 pre-built evaluations ready to run on any model", and its table of contents has Log Files, Inspect View and Task Views entries. The default log directory name and file layout were not verified verbatim in this pass.
关机后的任务Tasks after shutdown: 本机批量执行：官方文档目录列出 Running Eval Sets、Parallelism、Handling Errors、Setting Limits、Checkpointing 与 Intervention，即并行跑评测集、断点与人工介入是文档内的既有概念。关机后任务在别处续跑：官方未说明——它不是托管服务。
Local batch execution: the docs table of contents lists Running Eval Sets, Parallelism, Handling Errors, Setting Limits, Checkpointing and Intervention, i.e. parallel eval sets, checkpoints and human intervention are documented concepts. Whether a task continues elsewhere after your machine shuts down: not stated officially - Inspect is not a hosted service.
工具与扩展Tools & extensions: 官方文档首页列内置 bash、python、文本编辑、网页搜索、computer 工具，并写明支持自定义工具与 MCP 工具；文档目录另有 Tool Approval、Tool Result Review、Agent Bridge、Extensions 等条目。首页还称可运行外部 agent 作为被测对象，举例 Claude Code、Codex CLI、Gemini CLI。
The docs front page lists built-in bash, python, text editing, web search and computer tools, and states support for custom and MCP tools; the table of contents adds Tool Approval, Tool Result Review, Agent Bridge and Extensions. The same page describes running arbitrary external agents as the subject under test, naming Claude Code, Codex CLI and Gemini CLI.
上下文与记忆Context & memory: 文档目录列出 Compaction（上下文压缩）、Caching、Fallbacks、Multimodal、Structured Output，以及 Multiple Scorers、Scoring Policy、Model Grading 等评分侧概念。跨会话记忆不是它的模型：一次评测一份日志。压缩策略与默认阈值：本批次未取到官方正文。
The table of contents lists Compaction, Caching, Fallbacks, Multimodal and Structured Output on the model side, plus Multiple Scorers, Scoring Policy and Model Grading on the scoring side. Cross-session memory is not its model: one evaluation run, one log. Default compaction strategy and thresholds: no official text captured in this pass.
权限与限制Permissions & limits: MIT 许可（LICENSE 原文，版权方为 UK AI Security Institute）。运行在你自己的环境，模型调用走你配置的 provider 与 key；官方文档首页把沙箱系统说成支持在 Docker、Kubernetes、Modal、Proxmox、Vagrant 等「running untrusted model code」，并有扩展 API；文档目录另有 Sandboxing、Approvers、Hooks、Filesystems 条目。遥测与数据上传条款：本批次未取到官方说明。
MIT (LICENSE text read directly; copyright holder is the UK AI Security Institute). It runs in your environment and model calls go through the providers and keys you configure. The docs front page describes a sandboxing system for "running untrusted model code" in Docker, Kubernetes, Modal, Proxmox, Vagrant and other systems via an extension API, and the table of contents adds Sandboxing, Approvers, Hooks and Filesystems. Telemetry and data-upload terms: no official statement captured in this pass.
适合什么任务Best for: 适合：自己写评测与 agent 任务、要把不受信模型代码关进沙箱、要工具审批与本地日志复核、要跨 20 余家 provider 跑同一份任务。不适合：只想点托管控制台、只要一个指标库而不想维护 Python 评测、把跑分榜单当交付物。
Fits: writing your own evals and agent tasks, putting untrusted model code into a sandbox, keeping tool approval plus local logs for sample-by-sample review, and running one task definition across 20+ providers. Not a fit: wanting only a hosted console, needing a metric library without maintaining Python evaluations, or treating somebody else's leaderboard numbers as the deliverable.

头号误解Top misconception

把它当云端评测服务——沙箱是你自己提供的 Docker / K8s / Modal 等环境
以为评测靠 YAML 配就行——Inspect 的 Task 是写在 Python 里的
把「200+ 预置评测」当成现成结论——官方口径是可直接运行的实现，结果仍要你自己跑

证据来源Evidence sources

最近核验记录Latest recorded check 2026-11-06，内容更新日content updated on 2026-11-06。单项复核范围见正文或来源说明，不代表所有字段在同一天重新核验。See the text or source notes for item-specific scope; this does not mean every field was rechecked on that date.

类型Type	名称Name	链接Link
docs	Inspect · 官方文档首页（框架定位、模型 provider、沙箱、工具与 MCP、Inspect View；直读 200，2026-11-06）Inspect · official docs front page (positioning, model providers, sandboxing, tools and MCP, Inspect View; direct fetch returned 200, 2026-11-06)	https://inspect.aisi.org.uk/
changelog	Inspect · Changelog（文档首页导航给出的变更记录入口；本批次未逐条验读）Inspect · CHANGELOG page in the docs (entry point taken from the docs front page navigation; not reviewed item by item in this pass)	https://inspect.aisi.org.uk/CHANGELOG.html
repo	UKGovernmentBEIS/inspect_ai · 仓库 README（AI Security Institute 出品、200+ 预置评测、uv 开发流程；GitHub API 直读，2026-11-06）UKGovernmentBEIS/inspect_ai · repository README (AI Security Institute authorship, 200+ pre-built evaluations, uv development flow; read through the GitHub API, 2026-11-06)	https://github.com/UKGovernmentBEIS/inspect_ai
docs	UKGovernmentBEIS/inspect_ai · LICENSE（MIT 原文与版权行，GitHub API 直读，2026-11-06）UKGovernmentBEIS/inspect_ai · LICENSE (MIT text and copyright line, read through the GitHub API, 2026-11-06)	https://github.com/UKGovernmentBEIS/inspect_ai/blob/main/LICENSE

官方口径与未核到项Official record and unverified items

本站未做端到端实测：本档案没有运行 inspect eval，没有打开 Inspect View 复核日志，不产出分数、准确率或任何排名，也没有清点官方文档列出的预置评测清单。本节只组织官方文档首页与仓库 README 里出现过的说法（安装方式、模型 provider 与本地推理、沙箱后端、工具与 MCP、工具审批、日志与查看器、外部 agent 运行），逐条见 sources 与核验日。官方文档首页另有 Tutorial 与 Options 页，本批次未取到其正文。

This site has not completed end-to-end measured runs. inspect eval was not executed, Inspect View logs were not opened, and this entry produces no scores, accuracies or rankings, nor any audit of the pre-built evaluation list.

This section only organizes statements from the official docs front page and the repository README (installation, model providers and local inference, sandbox backends, tools and MCP, tool approval, logs and viewers, external agent runs), each tied to the sources list and the verification date. The docs Tutorial and Options pages were not captured on the day.

Suggested measurements (all pointing at things the official readable pages do not answer):

Cost and friction of running one task against a second provider, including fallback behaviour
The practical boundary between Tool Approval and a Docker sandbox when the model emits shell commands
Whether checkpoint/intervention resume works across two separate host runs

变更记录说明Change-record notes

本对象无独立读取的 changelog 正文：文档首页导航给出 CHANGELOG 入口（见 sources），本批次未逐条验读。维护节奏按上游仓库元数据判断：最近推送 2026-11-06（GitHub API 核验当日），仓库未被标记为 archived，故 lifecycle 记 active；但版本内具体变更内容未清点，属未核项。

No changelog text was read in this pass: the docs front page navigation exposes a CHANGELOG entry (listed in the sources), which was not reviewed item by item. Cadence comes from repository metadata: the last push is dated 2026-11-06 (GitHub API verified the same day) and the repository is not flagged as archived, so lifecycle is recorded as active; the concrete contents of each release were not inventoried and remain unverified.

第一次怎么用First run

先确定模型接入：官方快速上手示例是「装对应 SDK + 导出 API key」，本地推理走 HuggingFace / vLLM / SGLang。
pip install inspect-ai，用 inspect eval <task.py> --model <provider/model> 跑第一个任务。
需要模型执行代码或操作文件时，先读官方 Sandboxing 口径，把任务放进 Docker 等沙箱后端，再考虑工具审批。
想看逐样本结果，用官方列出的 Inspect View；要边写边调，官方推荐（但非必需）VS Code 扩展。
批量跑多任务时查官方文档的 Eval Sets 与 Parallelism 条目，而不是自己写循环。

Decide the model access first: the official quick-start pattern is "install the matching SDK + export the API key", and local inference goes through HuggingFace / vLLM / SGLang.
pip install inspect-ai, then run the first task with inspect eval <task.py> --model <provider/model>.
If the model executes code or touches files, read the official Sandboxing position first and put the task into a sandbox backend such as Docker, then consider tool approval.
To go through results sample by sample use the Inspect View tool listed in the docs; for authoring and debugging the official docs recommend (but do not require) the VS Code extension.
For batch runs, use the docs entries on Eval Sets and Parallelism rather than writing your own loop.