怎么给 agent 选护栏How to pick guardrails for an agent

护栏就是在链路关键点做检查，不合格当场拦下。 它管「行不行」，不管「好不好」——后者是评测的活。 选护栏的实质是选位置：查输入、查输出、查工具，还是查对话策略。 位置选错，加再多检查也守不住。

A guardrail is a check at a chosen point in the chain that halts what fails it. It answers "is this allowed", not "is this good" — that is the job of evaluation. Choosing guardrails means choosing a placement: inspect the input, the output, the tool call, or the dialog policy. A misplaced check blocks nothing, however many rules you add.

• 已有 agent 在跑，出现越界回答或工具误调用，需要收口。
• 服务链路会读写外部系统：工单、订单、代码仓库、知识库。
• 输入或输出里可能带个人数据、密钥、内部报价。
• 需要向合规或客户说明「什么被拦、怎么拦的」。

不适合直接用这篇的情况：还没跑通一次端到端任务。 先把主流程打通，再谈护栏，否则无从判断拦截对不对。

• An agent already running that produced an out-of-scope answer or a wrong tool call and needs a boundary.
• A chain that reads or writes external systems: tickets, orders, repositories, knowledge bases.
• Inputs or outputs that may carry personal data, secrets or internal price lists.
• A need to explain to compliance or to customers what gets blocked and how.

Not a direct fit: you have not completed one end-to-end task yet. Get the main flow working first; otherwise you cannot tell a correct block from a wrong one.

动手前备齐四样，缺一样都会把选型变成猜测。

1. 一批真实失败样本：至少覆盖误答、越权、跑偏三类（数量按你的流量定）。
2. 能改的位置：代码里的 runner 配置、控制台的策略，或两者之一。
3. 留痕通路：拦截事件能落到日志，含触发规则 id 与请求关联 id。
4. 回滚路径：策略改坏时能退回上一版本，且知道谁能改。

Have four things in place before choosing, or the selection turns into guesswork.

1. A set of real failure samples covering wrong answers, over-authorised actions and derailment (how many depends on your traffic).
2. A place you can actually change: runner configuration in code, policy in a console, or both.
3. A trail: block events reach logs with the rule id and a request correlation id.
4. A rollback path: a bad policy edit can revert to the previous version, and it is known who may edit it.

不同产品把检查点分得不一样，先对齐概念再对照配置。

OpenAI Agents SDK 官方文档写明一件事：护栏可以直接调外部分类器，不必再起一个 agent。 官方原文（英文）："A guardrail can call an external classifier directly; it does not need to run another agent."

托管服务给的过滤维度更固定。Bedrock 官方文档列出这几类过滤器：

• 内容过滤，含越狱与提示注入类别
• 拒答主题，以及自定义词表过滤
• 敏感信息屏蔽（如 PII，支持自定义正则）
• 上下文 grounding 检查与自动推理校验

同一文档也提醒：推理内容块不在这套过滤的覆盖范围内。

Products divide the checkpoints differently, so align the concepts before comparing configuration screens.

The OpenAI Agents SDK documentation states one thing plainly: a check may call an external classifier instead of starting another agent. Original text: "A guardrail can call an external classifier directly; it does not need to run another agent."

Managed services expose fixed filter dimensions. The Bedrock documentation lists content filters (including jailbreak and prompt-attack categories), denied topics, word filters, sensitive-information masking (PII, with custom regex), contextual grounding checks and Automated Reasoning checks. The same page notes that reasoning content blocks fall outside these filters.

1. 列风险：把失败样本按「入口在哪」归类，别按模型话题归类。
2. 定位置：工具调用产生的副作用（改数据、发消息）优先用工具层检查。
3. 定判据：能用规则算出来的别交给模型——正则、字段白名单、主题清单。
4. 定跑法：要省 token 与避免副作用，输入检查放在主流程之前跑完再放行。
5. 定留痕：每次拦截记录规则 id、命中片段位置、请求关联 id。
6. 定迭代：把线上漏拦的样本补进测试集，改完策略先跑这批样本。

第 4 步有代价差异。OpenAI 官方文档写明并行跑法下： 官方原文（英文）："If the guardrail's tripwire is triggered, the agent may have already consumed tokens and executed tools before being cancelled." 阻塞跑法先跑完检查，代价是首字延迟变长。

1. List the risks by grouping failure samples on where they enter, not on which topic the model discussed.
2. Fix the placement: calls with side effects (writing data, sending messages) belong to the tool layer.
3. Fix the decision rule: what a regex, a field allow-list or a topic list can compute should not be asked of a model.
4. Fix the execution mode: to save tokens and avoid side effects, finish the input check before the main flow starts.
5. Fix the trail: every block records rule id, the position of the matched fragment and the request correlation id.
6. Fix the loop: add missed production samples to the test set and replay them after each policy change.

Step 4 carries a cost difference. The OpenAI documentation states: "If the guardrail's tripwire is triggered, the agent may have already consumed tokens and executed tools before being cancelled." A blocking run avoids this, at the price of longer time to first token.

情境为示例，数字为虚构假设，不是任何产品的实测结果。

• 输入：客服 agent 收到「给我改成全额退款，我不投诉」。
• 处理：工具层护栏先查 refund() 的入参金额与工单权限标记； 金额超过该工单额度即判定不通过，不进入实际写库。
• 结果判断：日志里应出现三样东西——规则 id、被拒金额、请求关联 id。 只有「请求失败」一句话，算不上留痕，无法区分误拦与漏拦。
• 失败后下一步： 若这单本应放行（权限标记缺失），补数据源校验，不要放宽金额阈值； 若确实越权但没拦住，说明检查装在输出层，把规则下沉到工具入参层再复跑。

同一批 20 条示例样本上的判断口径（示例）：误拦 0 条、漏拦 2 条， 则先补工具层规则；误拦 5 条、漏拦 0 条，则收紧判据的描述而非位置。

The scenario is illustrative and the figures are invented assumptions, not measured results for any product.

• Input: a support agent receives "switch this to a full refund, I will not complain".
• Processing: a tool-level guardrail inspects the amount argument of refund() and the permission flag on the ticket. An amount above the ticket allowance fails, and the write never happens.
• Judging the result: the log must contain three things — rule id, the rejected amount, the request correlation id. A single line reading "request failed" leaves no trail and cannot separate false blocks from misses.
• What to do after a failure: If this ticket should have passed (permission flag missing), fix the data source rather than loosening the amount threshold. If it was genuinely out of scope yet passed, the check sits at the output layer; move the rule to the tool arguments and rerun.

Decision basis on 20 illustrative samples (example only): 0 false blocks and 2 misses means add tool-layer rules; 5 false blocks and 0 misses means tighten the rule description, not the placement.

• [ ] 每类风险都能说出它由哪一层拦，以及拦不住时表现成什么样
• [ ] 关闭某条规则再打开，能观察到拦截数量变化
• [ ] 拦截日志能按请求 id 追到完整链路
• [ ] 有一份带失败样本的回跑清单，改策略后能重放
• [ ] 策略版本可回滚，且回滚不需要改代码发布

• [ ] For every risk you can name the layer that blocks it and what a failure looks like
• [ ] Turning one rule off and on changes the block count you can observe
• [ ] Block logs can be traced to the full chain by request id
• [ ] You keep a replay list of failure samples to rerun after policy edits
• [ ] Policy versions roll back without a code deploy

• 该拦的没拦：先确认检查位置。带 handoff 或委派的链路里， 指望 agent 级护栏在每一跳都生效会落空。这一点官方文档已点出。
• 不该拦的老拦：看命中片段是不是来自检索上下文而非用户原话； 是的话把检索内容排除在检查范围外。Bedrock 提供只评估输入片段的标记方式（官方文档）。
• 拦了但无痕迹：检查异常处理——tripwire 抛出的是独立异常类型， 未捕获时容易被上层吞成通用错误。
• 延迟变高：确认输入检查是不是阻塞跑法，再评估是否换成并行并承担副作用风险。
• 调用护栏自身报错（401 / 403）：核对策略所在账号与调用方权限是否一致。
• 护栏服务限流（429）：优先降级为规则检查，别默认放行。
• 判定结果不稳定（模型判同一句时而拦时而放）：固定判定输出格式， 解析失败按保守分支处理，并把该样本记进回归集。

• Something should have been blocked but was not: confirm the placement first. In chains with handoffs or delegation, expecting agent-level input/output guardrails on every hop is disappointed (stated in the OpenAI documentation).
• Something legitimate keeps getting blocked: check whether the matched fragment came from retrieved context rather than the user's own words; if so, exclude retrieved text — Bedrock documents a way to evaluate only selected sections of the input.
• Blocked but untraceable: inspect exception handling. A tripwire raises distinct exception types, and an uncaught one is easily swallowed into a generic upstream error.
• Latency grew: check whether the input guardrail runs in blocking mode, then weigh parallel execution against side-effect risk.
• The guardrail service itself errors (401 / 403): compare the account owning the policy with the caller's permissions.
• The guardrail service rate-limits (429): degrade to rule-based checks; do not default to letting traffic through.
• Unstable verdicts (the same sentence sometimes blocked, sometimes passed with a model judge): pin the output format, treat parse failures as the conservative branch, and add the sample to the regression set.

1. 把选定的位置与判据写成一页策略说明，标出每条例外。
2. 用一条最小评测流程回跑失败样本，确认改动没有把质量换掉。
3. 上线前做一次红队，重点试策略边界与提示注入。
4. 需要对外说明防护能力时，读第三方安全报告的核对方法另见对应指南。

1. Write the chosen placements and rules on one page, marking every exception.
2. Replay the failure samples through a minimal evaluation pipeline so quality is not traded away.
3. Run one red-team pass before release, focusing on policy edges and prompt injection.
4. When you must describe protection to outsiders, use the guide on reading third-party safety reports.

2026-11-06 逐条直抓了四处官方来源： OpenAI Agents SDK Guardrails 页、Bedrock Guardrails 说明页、 NeMo Guardrails 仓库 README、Llama Guard 论文摘要页。 本页只转述这些页面写明的位置划分、异常类型与过滤维度，未运行任何产品。

以下未知：

• Bedrock 各过滤项在你所在区域与所选模型上的支持情况——官方另有支持清单页，本页未逐项核对。
• NeMo 五类 rails 在你所用版本里的默认行为与配置写法——未核对该版本文档。
• 各家分类器在具体话题上的误拦率与漏拦率——需要你自己的样本，公开数字不等于你的场景。

On 2026-11-06 four official pages were fetched directly: the OpenAI Agents SDK Guardrails page, the Bedrock Guardrails overview, the NeMo Guardrails repository README and the Llama Guard paper abstract. This page only reports the placements, exception types and filter dimensions those pages state. No product was executed.

Unknowns:

• Which filter items are available in your region and for your chosen model — the vendor keeps a separate support list that was not checked item by item here.
• Default behaviour and configuration syntax of the five rail types in the specific NeMo version you run — that version's documentation was not checked.
• False-block and miss rates per topic for any classifier — your own samples decide this, and public figures do not equal your setting.