← Safety

核验标记表示资料完整度，不是产品评分。下列未确认项可能影响选型；请按对应来源逐项确认。The confidence mark describes evidence completeness, not a product score. Unconfirmed items below may affect your choice; check them against the relevant sources.

模型与开放条件Model & openness

README 的指标示例把评判模型直接写进代码：AsyncOpenAI + llm_factory("gpt-4o", ...)，并提示需设置 OPENAI_API_KEY。 文档站的 LLM Providers 列 Amazon Bedrock、Google Gemini、OCI Gen AI，自定义面有 Customise models 与 LLM Adapters 页； 另有不需要模型的 Traditional non LLM metrics。逐 provider 支持矩阵：未逐条清点。

The judging model is written straight into the README's metric example: AsyncOpenAI plus llm_factory("gpt-4o", ...), with the note "Make sure your OPENAI_API_KEY environment variable is set". The docs site lists LLM Providers (Amazon Bedrock, Google Gemini, OCI Gen AI) and has Customise models and LLM Adapters pages, plus Traditional non LLM metrics that need no model at all. The per-provider support matrix was not inventoried.

运行位置Where it runs

pip install ragas 的 Python 库，评测在你自己进程的 evaluate() / aevaluate() 调用里跑，文档也有 CLI 一节； README 另给 ragas quickstart rag_eval -o ./my-project 拉模板工程。是否存在官方托管服务面：本批次未取到官方正文 （README 只给 Discord、博客与咨询邮箱）。

A pip install ragas Python library: evaluation runs inside your own process through evaluate() / aevaluate(), and the docs also have a CLI section. The README adds ragas quickstart rag_eval -o ./my-project to scaffold a template project. Whether any official hosted surface exists: no official text captured in this pass (the README offers only Discord, blog and a contact address).

本地文件Local files

输入是你自己的数据：文档把 Datasets、Evaluation Sample、Testset Generation（RAG / agents / persona / 单跳多跳查询）列为核心对象， 并称内置 dataset 管理与结果跟踪（Easy to integrate）。结果、缓存与生成测试集的中间产物落在哪里、默认是否上传：官方未说明。

The inputs are your own data: the docs treat Datasets, Evaluation Sample and Testset Generation (RAG, agents, personas, single- and multi-hop queries) as core objects, and the front page claims built-in dataset management and result tracking. Where results, caches and generated test sets land on disk, and whether anything is uploaded by default: not stated officially.

关机后的任务Tasks after shutdown

执行控制在库内：文档目录列出 Run Config、Executor、Caching、Cancelling Tasks，README 示例用 asyncio 调 metric.ascore(...)； 也就是说长跑靠你自己的进程与异步调用。关机后由谁接着跑、有无任务队列：官方未说明。

Execution control lives in the library: the docs table of contents lists Run Config, Executor, Caching and Cancelling Tasks, and the README example calls metric.ascore(...) through asyncio - long runs are your own process and your own async calls. Who resumes work after the machine shuts down, and whether a task queue exists: not stated officially.

工具与扩展Tools & extensions

扩展面是代码级：官方文档首页称可用简单装饰器创建自定义指标；集成面在文档目录里分三块—— Frameworks（LangChain、LangGraph、LlamaIndex、Haystack、AG-UI、Griptape、Swarm 等）、Observability（Arize、LangSmith）、LLM Providers。 MCP 接入方式：本批次未核验。

Extension is code-level: the docs front page says custom metrics can be created with simple decorators. The integration surface in the table of contents splits three ways - Frameworks (LangChain, LangGraph, LlamaIndex, Haystack, AG-UI, Griptape, Swarm and more), Observability (Arize, LangSmith) and LLM Providers. MCP integration method: not verified in this pass.

上下文与记忆Context & memory

「上下文」在 Ragas 里是被评对象，不是它自己的记忆：RAG 指标族含 Context Precision、Context Recall、Context Entities Recall、 Noise Sensitivity、Faithfulness、Response Relevancy（文档 Available Metrics 目录），另有 Evaluating Multi-turn Conversations 条目。 跨会话记忆机制：官方未说明。

"Context" in Ragas is the object being measured, not its own memory: the RAG metric family covers Context Precision, Context Recall, Context Entities Recall, Noise Sensitivity, Faithfulness and Response Relevancy (docs Available Metrics listing), and the site also lists Evaluating Multi-turn Conversations. Cross-session memory: not stated officially.

权限与限制Permissions & limits

Apache-2.0（LICENSE 原文直读）。README 单列 Open Analytics：称只收集最少量的匿名使用数据、采集代码在仓库内开源、汇总数据公开， 并写明「To opt-out, set the RAGAS_DO_NOT_TRACK environment variable to true」。 你的数据经模型 provider 后的留存与训练用途取决于所选 provider：本批次未取到官方条款。

Apache-2.0 (LICENSE text read directly). The README has an Open Analytics section stating that minimal, anonymized usage data is collected, that the collection code is open source in the repository, and that aggregated data is public, with "To opt-out, set the RAGAS_DO_NOT_TRACK environment variable to true". Retention and training use by the model provider your data passes through: no official terms captured in this pass.

适合什么任务Best for

适合：给 RAG 或 agent 应用配 LLM 评判指标、没有测试数据要先造测试集、要在自己代码与 CI 里建「改动—评测—观察—迭代」的循环。 不适合：想要一条 CLI 批量跑公开基准登记册、要沙箱与工具审批这类运行时边界、不愿为评判模型付出 API 调用成本。

Fits: attaching LLM-judged metrics to a RAG or agent application, generating a test set when you have none, and building a change-evaluate-observe-iterate loop inside your own code and CI. Not a fit: expecting one command to run a public benchmark registry and treating its output as a conclusion, needing sandboxing or tool approval as runtime boundaries, or refusing the API-call cost of judge models.

本页由This page is generated from the ai-handbook 数据层生成（CC BY 4.0）。data layer (CC BY 4.0). 方法论与坐标系定义见仓库内For the methodology and coordinate definitions, see METHODOLOGY.md.