From db33f44cbcd8085705b003a11aee94ed8defdadd Mon Sep 17 00:00:00 2001 From: Fu-Jie <33599649+Fu-Jie@users.noreply.github.com> Date: Thu, 26 Feb 2026 01:05:31 +0800 Subject: [PATCH] feat(plugins): release Copilot SDK Pipe v0.8.0 and Files Filter v0.1.3 (#50) * feat(plugins): release copilot sdk pipe v0.8.0 and files filter v0.1.3 - Add P1~P4 conditional tool filtering and admin/server gating behavior - Fix artifact publishing reliability, strict /api file URLs, and HTML preview/download delivery - Update bilingual README/docs, release notes, and filter matching/debug improvements * fix(docs): remove duplicate code block in tool-filtering zh doc - Remove incorrectly placed duplicate 'if not is_enabled: continue' block outside code fence on line 161-163 of copilot-sdk-tool-filtering.zh.md - Addresses review comment from gemini-code-assist (#50) --- README.md | 2 +- README_CN.md | 2 +- .../development/copilot-sdk-tool-filtering.md | 126 +++++ .../copilot-sdk-tool-filtering.zh.md | 206 +++++++ .../github-copilot-sdk-files-filter.md | 13 +- .../github-copilot-sdk-files-filter.zh.md | 13 +- docs/plugins/filters/index.md | 2 +- docs/plugins/filters/index.zh.md | 2 +- docs/plugins/pipes/github-copilot-sdk.md | 47 +- docs/plugins/pipes/github-copilot-sdk.zh.md | 45 +- docs/plugins/pipes/index.md | 2 +- docs/plugins/pipes/index.zh.md | 2 +- .../github_copilot_sdk_files_filter/README.md | 11 +- .../README_CN.md | 11 +- .../github_copilot_sdk_files_filter.py | 83 ++- plugins/pipes/github-copilot-sdk/README.md | 47 +- plugins/pipes/github-copilot-sdk/README_CN.md | 49 +- .../github-copilot-sdk/github_copilot_sdk.py | 501 +++++++++++------- plugins/pipes/github-copilot-sdk/v0.8.0.md | 127 +++++ plugins/pipes/github-copilot-sdk/v0.8.0_CN.md | 131 +++++ 20 files changed, 1175 insertions(+), 247 deletions(-) create mode 100644 docs/development/copilot-sdk-tool-filtering.md create mode 100644 docs/development/copilot-sdk-tool-filtering.zh.md create mode 100644 plugins/pipes/github-copilot-sdk/v0.8.0.md create mode 100644 plugins/pipes/github-copilot-sdk/v0.8.0_CN.md diff --git a/README.md b/README.md index ecf6fc4..c550ea6 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ A collection of enhancements, plugins, and prompts for [OpenWebUI](https://githu | Status | Plugin | Version | Downloads | Views | 📅 Updated | | :---: | :--- | :---: | :---: | :---: | :---: | -| 🆕 | [GitHub Copilot SDK Pipe](https://openwebui.com/posts/github_copilot_official_sdk_pipe_ce96f7b4) | ![v](https://img.shields.io/badge/v-0.7.0-blue?style=flat) | ![copilot_dl](https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2FFu-Jie%2Fdb3d95687075a880af6f1fba76d679c6%2Fraw%2Fbadge_post_aef940e01073e811a311c3a443d9c149_dl.json&style=flat) | ![copilot_vw](https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2FFu-Jie%2Fdb3d95687075a880af6f1fba76d679c6%2Fraw%2Fbadge_post_aef940e01073e811a311c3a443d9c149_vw.json&style=flat) | ![updated](https://img.shields.io/badge/2026--02--23-blue?style=flat) | +| 🆕 | [GitHub Copilot SDK Pipe](https://openwebui.com/posts/github_copilot_official_sdk_pipe_ce96f7b4) | ![v](https://img.shields.io/badge/v-0.8.0-blue?style=flat) | ![copilot_dl](https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2FFu-Jie%2Fdb3d95687075a880af6f1fba76d679c6%2Fraw%2Fbadge_post_aef940e01073e811a311c3a443d9c149_dl.json&style=flat) | ![copilot_vw](https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2FFu-Jie%2Fdb3d95687075a880af6f1fba76d679c6%2Fraw%2Fbadge_post_aef940e01073e811a311c3a443d9c149_vw.json&style=flat) | ![updated](https://img.shields.io/badge/2026--02--25-blue?style=flat) | ### 📈 Total Downloads Trend diff --git a/README_CN.md b/README_CN.md index eb51ccc..07ed86f 100644 --- a/README_CN.md +++ b/README_CN.md @@ -32,7 +32,7 @@ OpenWebUI 增强功能集合。包含个人开发与收集的插件、提示词 | 状态 | 插件 | 版本 | 下载 | 浏览 | 📅 更新 | | :---: | :--- | :---: | :---: | :---: | :---: | -| 🆕 | [GitHub Copilot SDK Pipe](https://openwebui.com/posts/github_copilot_official_sdk_pipe_ce96f7b4) | ![v](https://img.shields.io/badge/v-0.7.0-blue?style=flat) | ![copilot_dl](https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2FFu-Jie%2Fdb3d95687075a880af6f1fba76d679c6%2Fraw%2Fbadge_post_aef940e01073e811a311c3a443d9c149_dl.json&style=flat) | ![copilot_vw](https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2FFu-Jie%2Fdb3d95687075a880af6f1fba76d679c6%2Fraw%2Fbadge_post_aef940e01073e811a311c3a443d9c149_vw.json&style=flat) | ![updated](https://img.shields.io/badge/2026--02--23-blue?style=flat) | +| 🆕 | [GitHub Copilot SDK Pipe](https://openwebui.com/posts/github_copilot_official_sdk_pipe_ce96f7b4) | ![v](https://img.shields.io/badge/v-0.8.0-blue?style=flat) | ![copilot_dl](https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2FFu-Jie%2Fdb3d95687075a880af6f1fba76d679c6%2Fraw%2Fbadge_post_aef940e01073e811a311c3a443d9c149_dl.json&style=flat) | ![copilot_vw](https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2FFu-Jie%2Fdb3d95687075a880af6f1fba76d679c6%2Fraw%2Fbadge_post_aef940e01073e811a311c3a443d9c149_vw.json&style=flat) | ![updated](https://img.shields.io/badge/2026--02--25-blue?style=flat) | ### 📈 总下载量累计趋势 diff --git a/docs/development/copilot-sdk-tool-filtering.md b/docs/development/copilot-sdk-tool-filtering.md new file mode 100644 index 0000000..d414023 --- /dev/null +++ b/docs/development/copilot-sdk-tool-filtering.md @@ -0,0 +1,126 @@ +# GitHub Copilot SDK Tool Filtering Logic Documentation + +## Overview + +The tool filtering logic ensures that changes made in the **OpenWebUI admin panel take effect on the very next chat message** — no restart or cache flush required. The design balances three goals: administrator control, user autonomy, and built-in feature availability. + +## Priority Hierarchy + +Filtering is applied top-to-bottom. A higher layer can fully block a lower one: + +| Priority | Layer | Controls | +|---|---|---| +| 1 (Highest) | **Plugin Valve toggles** | `ENABLE_OPENWEBUI_TOOLS`, `ENABLE_MCP_SERVER`, `ENABLE_OPENAPI_SERVER` — category master switches | +| 2 | **Admin backend server toggle** | Per-server `config.enable` in OpenWebUI Connections panel — blocks specific servers | +| 3 (Lowest) | **User Chat menu selection** | `tool_ids` from the chat UI — selects which enabled items to use | + +--- + +## Core Decision Logic (Flowchart) + +```mermaid +graph TD + A[New message arrives] --> V{Plugin Valve enabled\nfor this category?} + V -- No --> VX[Drop all tools in this category] + V -- Yes --> B{Admin backend:\nconfig.enable = True?} + B -- No --> C[Skip this server] + B -- Yes --> F{Built-in or Custom/Server tool?} + + F -- Built-in --> G{Any builtin: IDs\nselected in Chat?} + G -- Yes --> H[Enable ONLY the mapped categories\nunselected categories set to False] + G -- No --> I[Enable default 4 categories:\nweb_search, image_generation,\ncode_interpreter, memory] + + F -- Custom / Server --> J{Any custom IDs\nselected in Chat?} + J -- Yes --> K[Load ONLY the selected IDs] + J -- No --> L[Load ALL admin-enabled custom tools] + + H & I & K & L --> M[Always inject: publish_file_from_workspace] + M --> N[Start / Resume Copilot SDK Session] +``` + +--- + +## Scenario Reference Table + +| User selects in Chat | Custom tools loaded | Built-in tools loaded | +|---|---|---| +| Nothing | All admin-enabled | Default 4 (search, image, code, memory) | +| Only `builtin:xxx` | All admin-enabled (unaffected) | Only selected categories | +| Only custom/server IDs | Only selected IDs | Default 4 | +| Both builtin and custom | Only selected custom IDs | Only selected builtin categories | + +--- + +## Technical Implementation Details + +### 1. Real-time Admin Sync (No Caching) + +Every request re-reads `TOOL_SERVER_CONNECTIONS.value` live. There is **no in-memory cache** for server state. As a result: + +- Enable a server in the admin panel → it appears on the **next message**. +- Disable a server → it is dropped on the **next message**. + +```python +# Read live on every request — no cache +if hasattr(TOOL_SERVER_CONNECTIONS, "value"): + raw_connections = TOOL_SERVER_CONNECTIONS.value + +for server in connections: + is_enabled = config.get("enable", False) # checked per-server, per-request + if not is_enabled: + continue # skipped immediately — hard block +``` + +### 2. Built-in Tool Category Mapping + +The plugin maps individual `builtin:func_name` IDs to one of 9 categories understood by `get_builtin_tools`. When the user selects specific builtins, **only those categories are enabled; unselected categories are explicitly set to `False`** (not omitted) to prevent OpenWebUI's default-`True` fallback: + +```python +if builtin_selected: + # Strict mode: set every category explicitly + for cat in all_builtin_categories: # all 9 + is_enabled = cat in enabled_categories # only selected ones are True + builtin_tools_meta[cat] = is_enabled # unselected are explicitly False +else: + # Default mode: only the 4 core categories + default_builtin_categories = [ + "web_search", "image_generation", "code_interpreter", "memory" + ] + for cat in all_builtin_categories: + builtin_tools_meta[cat] = cat in default_builtin_categories + features.update(req_features) # merge backend feature flags +``` + +### 3. Custom Tool "Select-All" Fallback + +The whitelist is activated **only when the user explicitly selects custom/server IDs**. Selecting only `builtin:` IDs does not trigger the custom whitelist, so all admin-enabled servers remain accessible: + +```python +# custom_selected contains only non-builtin: IDs +if custom_selected: + # Whitelist active: keep only what the user picked + tool_ids = [tid for tid in available_ids if tid in custom_selected] +else: + # No custom selection: load everything enabled in backend + tool_ids = available_ids +``` + +The same rule applies to MCP servers in `_parse_mcp_servers`. + +### 4. Admin Backend Strict Validation + +Applied uniformly to both OpenAPI and MCP servers, handling both dict and Pydantic object shapes: + +```python +is_enabled = False +config = server.get("config", {}) if isinstance(server, dict) else getattr(server, "config", {}) +is_enabled = config.get("enable", False) if isinstance(config, dict) else getattr(config, "enable", False) + +if not is_enabled: + continue # hard skip — no user or valve setting can override this +``` + +## Important Notes + +- **SDK Internal Tools**: `available_tools = None` is passed to the session so SDK-native capabilities (`read_file`, `shell`, etc.) are never accidentally blocked by the custom tool list. +- **Persistent Tool**: `publish_file_from_workspace` is always injected after all filtering — it is required for the file delivery workflow regardless of any toggle. diff --git a/docs/development/copilot-sdk-tool-filtering.zh.md b/docs/development/copilot-sdk-tool-filtering.zh.md new file mode 100644 index 0000000..a8605cd --- /dev/null +++ b/docs/development/copilot-sdk-tool-filtering.zh.md @@ -0,0 +1,206 @@ +# GitHub Copilot SDK 工具过滤逻辑开发文档 + +## 核心需求 + +**管理员在后台修改工具服务的启用状态后，用户发送下一条消息时立即生效，无需重启服务或刷新缓存。** + +过滤逻辑同时兼顾两个目标：管理员管控权、用户自主选择权。内置工具则完全独立，仅由模型配置决定。 + +--- + +## 工具分类说明 + +本文档涉及两类完全独立的工具，权限控制机制不同： + +| 工具类型 | 说明 | 权限控制来源 | +|---|---|---| +| **内置工具（Builtin Tools）** | OpenWebUI 原生能力：时间、知识库、记忆、联网搜索、图像生成、代码解释器等 | 仅由模型配置 `meta.builtinTools` 决定，**与 Chat 前端选择无关** | +| **OpenWebUI Tools** | 用户安装的 Python 工具插件 | 插件 Valve + Chat 工具选择（tool_ids） | +| **工具服务器（OpenAPI / MCP）** | 外部 OpenAPI Server、MCP Server | 插件 Valve + 管理员 `config.enable` + `function_name_filter_list` + Chat 工具选择（tool_ids） | + +--- + +## 内置工具权限控制（模型配置驱动，与前端无关） + +内置工具**完全由模型配置决定**，Chat 界面的工具选择对其没有任何影响。 + +### 模型 `meta.builtinTools` 字段 + +在模型（自定义模型或基础模型）的 `meta` 字段中有一个可选的 `builtinTools` 对象： + +```json +{ + "meta": { + "capabilities": { "builtin_tools": true }, + "builtinTools": { + "time": false, + "memory": true, + "chats": true, + "notes": true, + "knowledge": true, + "channels": true, + "web_search": true, + "image_generation": true, + "code_interpreter": true + } + } +} +``` + +**判定规则（源码 `utils/tools.py`）：** + +```python +def is_builtin_tool_enabled(category: str) -> bool: + builtin_tools = model.get("info", {}).get("meta", {}).get("builtinTools", {}) + return builtin_tools.get(category, True) # 缺省时默认 True +``` + +- `builtinTools` 字段**不存在** → 所有内置工具类别默认全部开启 +- `builtinTools` 字段**存在** → 仅值为 `true` 的类别开启，其余关闭 + +--- + +## OpenWebUI Tools 和工具服务器的优先级层级 + +这两类工具的过滤从上到下依次执行，**受 Chat 前端选择影响**： + +| 优先级 | 层级 | 控制范围 | +|---|---|---| +| 1（最高） | **插件 Valve 开关** | `ENABLE_OPENWEBUI_TOOLS` / `ENABLE_MCP_SERVER` / `ENABLE_OPENAPI_SERVER` — 类别总开关 | +| 2 | **管理员后端服务器开关** | OpenWebUI 连接面板中每个服务器的 `config.enable` — 控制具体服务器是否启用 | +| 3 | **管理员函数名过滤列表** | 工具服务器 `config.function_name_filter_list` — 限制该服务器对外暴露的函数列表（逗号分隔） | +| 4（最低） | **用户 Chat 工具选择** | Chat 界面的 `tool_ids` — 在已启用范围内进一步筛选：未选则全选，有选则仅选中的 | + +### 管理员函数名过滤列表说明 + +OpenWebUI 后台的工具服务器连接配置中支持设置 `function_name_filter_list` 字段（逗号分隔的函数名），用于限制该服务器对外暴露的函数。源码逻辑： + +```python +# utils/tools.py +function_name_filter_list = tool_server_connection.get("config", {}).get("function_name_filter_list", "") +if isinstance(function_name_filter_list, str): + function_name_filter_list = function_name_filter_list.split(",") + +for spec in specs: + function_name = spec["name"] + if function_name_filter_list: + if not is_string_allowed(function_name, function_name_filter_list): + continue # 不在列表中的函数被跳过 +``` + +- 列表**为空** → 该服务器所有函数均可用 +- 列表**有值** → 只有名称匹配的函数会被暴露给用户 + +--- + +## 核心判定流程 + +```mermaid +graph TD + A[新消息到来] --> BT[内置工具：读模型 meta.builtinTools] + BT --> BT2{builtinTools 字段存在?} + BT2 -- 否 --> BT3[开启全部内置工具] + BT2 -- 是 --> BT4[仅开启值为 true 的类别] + + A --> CT[OpenWebUI Tools / 工具服务器] + CT --> V{插件 Valve 开启了该类别?} + V -- 否 --> VX[丢弃该类别] + V -- 是 --> B{后端 config.enable = True?} + B -- 否 --> C[跳过该服务器] + B -- 是 --> FL{function_name_filter_list 有值?} + FL -- 是 --> FL2[过滤掉不在列表中的函数] + FL -- 否 --> J + FL2 --> J{Chat tool_ids 有勾选?} + J -- 有 --> K[仅加载勾选的 ID] + J -- 无 --> L[加载所有后台已启用工具] + + BT3 & BT4 & K & L --> M[始终注入: publish_file_from_workspace] + M --> N[启动/恢复 Copilot SDK 会话] +``` + +--- + +## 场景速查表 + +### 内置工具（与前端选择无关） + +| 模型配置 | 结果 | +|---|---| +| `meta.builtinTools` 字段不存在 | 全部内置工具类别开启 | +| `meta.builtinTools` 字段存在 | 仅 `true` 的类别开启 | + +### OpenWebUI Tools / 工具服务器（受 Chat 前端选择影响） + +| Chat 工具选择情况 | 加载逻辑 | +|---|---| +| 什么都没选 | 加载所有 Valve 开启且后台已启用的工具（Python Tools + OpenAPI + MCP） | +| 选了部分 tool_ids | 仅加载勾选的 ID（必须同时通过 Valve 和 config.enable 校验） | + +--- + +## 代码实现详述 + +### 1. 管理员后台变更即时同步 + +OpenWebUI 通过 `PersistentConfig` + Redis 保证多 worker 之间的配置同步，插件直接读取 `request.app.state.config.TOOL_SERVER_CONNECTIONS` 即可获取最新值： + +- 后台**启用**一个服务器 → **下一条消息**就出现。 +- 后台**禁用**一个服务器 → **下一条消息**就消失。 + +```python +# 直接读取 OpenWebUI 的配置对象，有 Redis 时每次读取都会同步最新值 +connections = request.app.state.config.TOOL_SERVER_CONNECTIONS # list + +for server in connections: + config = server.get("config", {}) + is_enabled = config.get("enable", False) # 每条服务器、每次请求都检查 + if not is_enabled: + continue # 立即跳过，硬性拦截 +``` + +### 2. 内置工具直接透传给 OpenWebUI 处理 + +插件调用 OpenWebUI 的 `get_builtin_tools(request, extra_params, model)` 时，将 `model` 原样传入即可。OpenWebUI 内部会自动读取 `model.info.meta.builtinTools` 来决定哪些内置工具生效： + +```python +# OpenWebUI 源码 utils/tools.py 中的判定逻辑（开发参考，非插件代码） +def is_builtin_tool_enabled(category: str) -> bool: + builtin_tools = model.get("info", {}).get("meta", {}).get("builtinTools", {}) + return builtin_tools.get(category, True) # 缺省值 True：未配置时全部开启 +``` + +插件无需自行维护内置工具分类映射，也不需要向 `builtinTools` 注入任何值。 + +### 3. 自定义工具"默认全选"（白名单仅在显式勾选时激活） + +白名单**只有在用户明确勾选了 tool_ids 时才启用**。未勾选任何工具时，加载所有后台已启用工具： + +```python +# tool_ids 来自 Chat 请求体 +if tool_ids: + # 白名单模式：严格只保留勾选项 + available_ids = [tid for tid in available_ids if tid in tool_ids] +else: + # 无勾选：加载所有后台已启用工具（免配置可用） + pass # available_ids 保持不变 +``` + +MCP 服务器的 `_parse_mcp_servers` 遵循同样规则。 + +### 4. 后端状态硬校验（OpenAPI 和 MCP 统一处理） + +```python +is_enabled = False +config = server.get("config", {}) if isinstance(server, dict) else getattr(server, "config", {}) +is_enabled = config.get("enable", False) if isinstance(config, dict) else getattr(config, "enable", False) + +if not is_enabled: + continue # 硬性跳过，任何用户或 Valve 设置都无法绕过 +``` + +--- + +## 注意事项 + +- **SDK 内部工具**：向会话传 `available_tools = None`，保证 `read_file`、`shell` 等 SDK 原生能力不被自定义工具列表意外屏蔽。 +- **始终注入的工具**：`publish_file_from_workspace` 在所有过滤完成后硬性追加，是文件交付工作流的必要依赖，不受任何开关影响。 diff --git a/docs/plugins/filters/github-copilot-sdk-files-filter.md b/docs/plugins/filters/github-copilot-sdk-files-filter.md index 12c2f84..e4e63a1 100644 --- a/docs/plugins/filters/github-copilot-sdk-files-filter.md +++ b/docs/plugins/filters/github-copilot-sdk-files-filter.md @@ -1,9 +1,14 @@ -# GitHub Copilot SDK Files Filter (v0.1.2) +# GitHub Copilot SDK Files Filter (v0.1.3) This is a dedicated **companion filter plugin** designed specifically for the [GitHub Copilot SDK Pipe](../pipes/github-copilot-sdk.md). Its core mission is to **protect user-uploaded files from being "pre-processed" by the OpenWebUI core system, ensuring that the Copilot Agent receives the raw files for autonomous analysis.** +## ✨ v0.1.3 Updates (What's New) + +- **🔍 BYOK Model ID Matching Fixed**: Now correctly identifies models in `github_copilot_official_sdk_pipe.xxx` format via prefix matching, in addition to keyword fallback for backward compatibility. (v0.1.3) +- **🐛 Dual-channel Debug Log**: Added `show_debug_log` valve. When enabled, logs are written to both server-side logger and browser console (`console.group`). (v0.1.3) + ## 🎯 Why is this needed? In OpenWebUI's default workflow, when you upload a file (e.g., PDF, Excel, Python script), OpenWebUI automatically initiates a **RAG (Retrieval-Augmented Generation)** process: parsing the file, vectorizing it, extracting text, and injecting it into the prompt. @@ -48,6 +53,6 @@ Default settings work for most users unless you have specific needs: ## ⚠️ Important Notes -* **Must be used with Copilot SDK Pipe**: If you install this plugin without the main Pipe plugin, uploaded files will simply "disappear" (as no subsequent plugin will look for them in `copilot_files`). -* **Gemini Filter Compatibility**: This plugin is fully compatible with the Gemini Multimodal Filter. As long as the priority is set correctly (This Plugin < Gemini Plugin), they can coexist without interference. -* **Physical File Path**: Ensure the `OPENWEBUI_UPLOAD_PATH` is correctly set in the Pipe plugin Valves for the actual file transport to work. +- **Must be used with Copilot SDK Pipe**: If you install this plugin without the main Pipe plugin, uploaded files will simply "disappear" (as no subsequent plugin will look for them in `copilot_files`). +- **Gemini Filter Compatibility**: This plugin is fully compatible with the Gemini Multimodal Filter. As long as the priority is set correctly (This Plugin < Gemini Plugin), they can coexist without interference. +- **Physical File Path**: Ensure the `OPENWEBUI_UPLOAD_PATH` is correctly set in the Pipe plugin Valves for the actual file transport to work. diff --git a/docs/plugins/filters/github-copilot-sdk-files-filter.zh.md b/docs/plugins/filters/github-copilot-sdk-files-filter.zh.md index 615c50c..646df69 100644 --- a/docs/plugins/filters/github-copilot-sdk-files-filter.zh.md +++ b/docs/plugins/filters/github-copilot-sdk-files-filter.zh.md @@ -1,9 +1,14 @@ -# GitHub Copilot SDK 文件过滤器 (v0.1.2) +# GitHub Copilot SDK 文件过滤器 (v0.1.3) 这是一个专门为 [GitHub Copilot SDK Pipe](../pipes/github-copilot-sdk.zh.md) 设计的**伴侣过滤器插件**。 它的核心使命是：**保护用户上传的文件不被 OpenWebUI 核心系统“抢先处理”，确保 Copilot Agent 能够接收到原始文件并进行自主分析。** +## ✨ 0.1.3 更新内容 (What's New) + +- **🔍 BYOK 模型 ID 匹配修复**: 新增前缀匹配（`github_copilot_official_sdk_pipe.xxx` 格式），修复 BYOK 模型无法被正确识别的问题，关键词兜底保持向后兼容。(v0.1.3) +- **🐛 双通道调试日志**: 新增 `show_debug_log` 配置项，启用后同时向后端日志和浏览器控制台（`console.group`）输出调试信息。(v0.1.3) + ## 🎯 为什么需要它？ 在 OpenWebUI 的默认流程中，当你上传一个文件（如 PDF、Excel、Python 脚本）时，OpenWebUI 会自动启动 **RAG（检索增强生成）** 流程：解析文件、向量化、提取文本并注入到提示词中。 @@ -48,6 +53,6 @@ ## ⚠️ 注意事项 -* **必须配合 Copilot SDK Pipe 使用**：如果你没有安装主 Pipe 插件，本插件将导致上传的文件“凭空消失”（因为没有后续插件去 `copilot_files` 里找它们）。 -* **Gemini Filter 兼容性**：本插件已完美兼容 Gemini 多模态过滤器。只要优先级设置正确（本插件 < Gemini 插件），它们可以共存互不干扰。 -* **物理文件路径**: 确保在 Pipe 插件的 Valves 中正确设置了 `OPENWEBUI_UPLOAD_PATH`，以便文件自动搬运功能正常工作。 +- **必须配合 Copilot SDK Pipe 使用**：如果你没有安装主 Pipe 插件，本插件将导致上传的文件“凭空消失”（因为没有后续插件去 `copilot_files` 里找它们）。 +- **Gemini Filter 兼容性**：本插件已完美兼容 Gemini 多模态过滤器。只要优先级设置正确（本插件 < Gemini 插件），它们可以共存互不干扰。 +- **物理文件路径**: 确保在 Pipe 插件的 Valves 中正确设置了 `OPENWEBUI_UPLOAD_PATH`，以便文件自动搬运功能正常工作。 diff --git a/docs/plugins/filters/index.md b/docs/plugins/filters/index.md index ab66a60..164a6af 100644 --- a/docs/plugins/filters/index.md +++ b/docs/plugins/filters/index.md @@ -82,7 +82,7 @@ Filters act as middleware in the message pipeline: A specialized filter to bypass OpenWebUI's default RAG for GitHub Copilot SDK models. It ensures the Agent receives raw files for autonomous analysis. - **Version:** 0.1.2 + **Version:** 0.1.3 [:octicons-arrow-right-24: Documentation](github-copilot-sdk-files-filter.md) diff --git a/docs/plugins/filters/index.zh.md b/docs/plugins/filters/index.zh.md index cfbe491..4894ea6 100644 --- a/docs/plugins/filters/index.zh.md +++ b/docs/plugins/filters/index.zh.md @@ -62,7 +62,7 @@ Filter 充当消息管线中的中间件： 专门用于绕过 OpenWebUI 默认 RAG 机制的过滤器，针对 GitHub Copilot SDK 模型。确保 Agent 能够接收到原始文件进行自主分析。 - **版本：** 0.1.2 + **版本：** 0.1.3 [:octicons-arrow-right-24: 查看文档](github-copilot-sdk-files-filter.zh.md) diff --git a/docs/plugins/pipes/github-copilot-sdk.md b/docs/plugins/pipes/github-copilot-sdk.md index 6cf091e..5b182e1 100644 --- a/docs/plugins/pipes/github-copilot-sdk.md +++ b/docs/plugins/pipes/github-copilot-sdk.md @@ -1,6 +1,6 @@ # GitHub Copilot SDK Pipe for OpenWebUI -**Author:** [Fu-Jie](https://github.com/Fu-Jie) | **Version:** 0.7.0 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT +**Author:** [Fu-Jie](https://github.com/Fu-Jie) | **Version:** 0.8.0 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT This is an advanced Pipe function for [OpenWebUI](https://github.com/open-webui/open-webui) that integrates the official [GitHub Copilot SDK](https://github.com/github/copilot-sdk). It enables you to use **GitHub Copilot models** (e.g., `gpt-5.2-codex`, `claude-sonnet-4.5`,`gemini-3-pro`, `gpt-5-mini`) **AND** your own models via **BYOK** (OpenAI, Anthropic) directly within OpenWebUI, providing a unified agentic experience with **strict User & Chat-level Workspace Isolation**. @@ -14,13 +14,23 @@ This is an advanced Pipe function for [OpenWebUI](https://github.com/open-webui/ --- -## ✨ v0.7.0 Updates (What's New) +## ✨ v0.8.0 Updates (What's New) -- **🚀 Integrated CLI Management**: The Copilot CLI is now automatically managed and bundled via the `github-copilot-sdk` pip package. (v0.7.0) -- **🧠 Native Tool Call UI**: Full adaptation to **OpenWebUI's native tool call UI** and thinking process visualization. (v0.7.0) -- **🏠 OpenWebUI v0.8.0+ Fix**: Resolved "Error getting file content" download failure by switching to absolute path registration for published files. (v0.7.0) -- **🌐 Comprehensive Multi-language Support**: Native localization for status messages in 11 languages (EN, ZH, JA, KO, FR, DE, ES, IT, RU, VI, ID). (v0.7.0) -- **🧹 Architecture Cleanup**: Refactored core setup and optimized reasoning status display for a leaner experience. (v0.7.0) +- **🎛️ Conditional Tool Filtering (P1~P4)**: Four-priority tool permission system. **Default ON**: If no tools are selected in Chat UI (P4), all enabled tools are active. **Whitelist Mode**: Once specific tools are checked, the whitelist strictly filters both OpenWebUI tools and MCP servers. Admin-level `config.enable` (P2) allows global server disabling. (v0.8.0) +- **🔧 File Publish Reliability**: Fixed `Error getting file content` across all storage backends (local/S3/GCS/Azure) by using `Storage.upload_file()` directly in the fallback path. HTML files are no longer blocked by `ALLOWED_FILE_EXTENSIONS` (`?process=false` always applied). (v0.8.0) +- **🌐 HTML Direct Access Link**: When `publish_file_from_workspace` publishes an HTML file, the plugin also provides a directly accessible HTML link for instant in-chat preview/opening. (v0.8.0) +- **🔒 Strict File URL Format**: Published file links must be relative paths starting with `/api/v1/files/` (e.g., `/api/v1/files/{id}/content/html`). Do not use `api/...` and do not prepend any domain. (v0.8.0) +- **🛠️ CLI Built-in Tools Always Available**: `available_tools` is now always `None`, ensuring Copilot CLI built-ins (e.g. `bash`, `create_file`) are never silently blocked regardless of MCP configuration. (v0.8.0) +- **📌 Publish Tool Always Injected**: `publish_file_from_workspace` is no longer lost when `ENABLE_OPENWEBUI_TOOLS` is disabled. (v0.8.0) +- **⚠️ Code Interpreter Limitation**: The `code_interpreter` tool runs in a remote, ephemeral environment. A system prompt warning now clarifies that it cannot access local files or persist changes. (v0.8.0) + +### 🐞 Bug Fixes in v0.8.0 + +- Fixed `{"detail":"[ERROR: Error getting file content]"}` when publishing files under object storage backends by replacing fallback manual copy/DB writes with `Storage.upload_file()`. +- Fixed HTML artifact upload being rejected by `ALLOWED_FILE_EXTENSIONS` by always appending `?process=false` on file upload API calls. +- Fixed invalid artifact links generated as `api/...` or domain-prefixed absolute URLs; links are now constrained to `/api/v1/files/...` relative paths. +- Fixed Copilot CLI built-ins being silently unavailable when no server tools were configured/loaded (which resulted in `available_tools=[]`); now `available_tools` remains `None`. +- Fixed `publish_file_from_workspace` disappearing when `ENABLE_OPENWEBUI_TOOLS` was disabled. --- @@ -33,11 +43,23 @@ This is an advanced Pipe function for [OpenWebUI](https://github.com/open-webui/ - **🧠 Deep Database Integration**: Real-time persistence of TOD·O lists for long-running workflows. - **🌊 Advanced Streaming**: Full support for thinking process/Chain of Thought visualization. - **🖼️ Intelligent Multimodal**: Vision capabilities and raw file analysis support (bypasses RAG for direct binary access). -- **📤 Workspace Artifacts (`publish_file_from_workspace`)**: Agents can generate files (Excel, CSV, HTML reports, etc.) and provide **persistent download links** directly in the chat. +- **📤 Workspace Artifacts (`publish_file_from_workspace`)**: Agents can generate files (Excel, CSV, HTML reports, etc.) and provide **persistent download links** directly in the chat. For HTML files, a direct-access HTML link is also provided. - **🖼️ Interactive Artifacts**: Automatically renders HTML/JS apps generated by the agent directly in the chat interface. --- +## 🧩 Companion Files Filter (Required for raw files) + +`GitHub Copilot SDK Files Filter` is the companion plugin that prevents OpenWebUI's default RAG pre-processing from consuming uploaded files before the Pipe receives them. + +- **What it does**: Moves uploaded files to `copilot_files` so the Pipe can access raw binaries directly. +- **Why it matters**: Without it, uploaded files may be parsed/vectorized early and the Agent may lose direct raw-file access. +- **v0.1.3 highlights**: + - BYOK model-id matching fix (supports `github_copilot_official_sdk_pipe.xxx` prefixes). + - Optional dual-channel debug log (`show_debug_log`) to backend logger + browser console. + +--- + ## ⚙️ Core Configuration (Valves) ### 1. Administrator Settings (Base) @@ -105,6 +127,15 @@ If this plugin has been useful, a **Star** on [OpenWebUI Extensions](https://git 2. Create **Fine-grained token**, granting **Account permissions** -> **Copilot Requests** access. 3. Paste the generated Token into the `GH_TOKEN` field in Valves. +### 3) Authentication Requirement (Mandatory) + +You MUST configure **at least one** credential source: + +- `GH_TOKEN` (GitHub Copilot subscription route), or +- `BYOK_API_KEY` (OpenAI/Anthropic route). + +If neither is configured, the model list will not appear. + --- ## 📋 Troubleshooting & Dependencies diff --git a/docs/plugins/pipes/github-copilot-sdk.zh.md b/docs/plugins/pipes/github-copilot-sdk.zh.md index 5f620be..6a4e6e5 100644 --- a/docs/plugins/pipes/github-copilot-sdk.zh.md +++ b/docs/plugins/pipes/github-copilot-sdk.zh.md @@ -1,6 +1,6 @@ # GitHub Copilot SDK 官方管道 -**作者:** [Fu-Jie](https://github.com/Fu-Jie) | **版本:** 0.7.0 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT +**作者:** [Fu-Jie](https://github.com/Fu-Jie) | **版本:** 0.8.0 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT 这是一个用于 [OpenWebUI](https://github.com/open-webui/open-webui) 的高级 Pipe 函数，深度集成了 **GitHub Copilot SDK**。它不仅支持 **GitHub Copilot 官方模型**（如 `gpt-5.2-codex`, `claude-sonnet-4.5`, `gemini-3-pro`, `gpt-5-mini`），还支持 **BYOK (自带 Key)** 模式对接自定义服务商（OpenAI, Anthropic），并具备**严格的用户与会话级工作区隔离**能力，提供统一且安全的 Agent 交互体验。 @@ -14,13 +14,23 @@ --- -## ✨ 0.7.0 更新内容 (What's New) +## ✨ 0.8.0 更新内容 (What's New) -- **🚀 CLI 免维护集成**: Copilot CLI 现在通过 `github-copilot-sdk` pip 包自动同步管理，彻底告别手动 `curl | bash` 安装问题。(v0.7.0) -- **🧠 原生工具调用 UI**: 全面适配 **OpenWebUI 原生工具调用 UI** 与模型思考过程（思维链）展示。(v0.7.0) -- **🏠 OpenWebUI v0.8.0+ 兼容性修复**: 通过切换为绝对路径注册发布文件，彻底解决了“Error getting file content”无法下载到本地的问题。(v0.7.0) -- **🌐 全面的多语言支持**: 针对状态消息进行了 11 国语言的原生本地化 (中/英/日/韩/法/德/西/意/俄/越/印尼)。(v0.7.0) -- **🧹 架构精简**: 重构了初始化逻辑并优化了推理状态显示，提供更轻量稳健的体验。(v0.7.0) +- **🎛️ 条件工具过滤 (P1~P4)**: 四优先级工具权限体系。**默认全开**: 若未在 Chat UI (P4) 勾选任何工具，则默认启用所有工具；**白名单模式**: 一旦勾选特定工具，即刻进入严格过滤模式，且 MCP server 同步受控；管理员亦可通过 `config.enable` (P2) 全局禁用工具服务器。(v0.8.0) +- **🔧 文件发布全面修复**: 通过在回退路径直接调用 `Storage.upload_file()`，彻底修复了所有存储后端（local/S3/GCS/Azure）下的 `Error getting file content` 问题；同时上传时自动携带 `?process=false`，HTML 文件不再被 `ALLOWED_FILE_EXTENSIONS` 拦截。(v0.8.0) +- **🌐 HTML 直达链接**: 当 `publish_file_from_workspace` 发布的是 HTML 文件时，插件会额外提供可直接访问的 HTML 链接，便于在聊天中即时预览/打开。(v0.8.0) +- **🔒 文件链接格式严格约束**: 发布链接必须是以 `/api/v1/files/` 开头的相对路径（例如 `/api/v1/files/{id}/content/html`）。禁止使用 `api/...`，也禁止拼接任何域名。(v0.8.0) +- **🛠️ CLI 内置工具始终可用**: `available_tools` 统一设为 `None`，Copilot CLI 内置工具（如 `bash`、`create_file`）无论 MCP 配置如何都不会被静默屏蔽。(v0.8.0) +- **📌 发布工具始终注入**: 即使 `ENABLE_OPENWEBUI_TOOLS` 关闭，`publish_file_from_workspace` 工具也不再丢失。(v0.8.0) +- **⚠️ 代码解释器限制**: `code_interpreter` 工具运行在远程临时环境中。系统提示词现已包含警告，明确指出该工具无法访问本地文件或持久化更改。(v0.8.0) + +### 🐞 v0.8.0 Bug 修复说明 + +- 修复了对象存储后端发布文件时出现的 `{"detail":"[ERROR: Error getting file content]"}`，回退路径从手动复制/写库改为 `Storage.upload_file()`。 +- 修复了 HTML 产物被 `ALLOWED_FILE_EXTENSIONS` 拦截的问题，上传接口统一追加 `?process=false`。 +- 修复了产物链接偶发被生成成 `api/...` 或带域名绝对 URL 的问题，现统一限制为 `/api/v1/files/...` 相对路径。 +- 修复了在未配置/未加载任何 server 工具时（最终出现 `available_tools=[]`）Copilot CLI 内置工具被静默禁用的问题，现统一保持 `available_tools=None`。 +- 修复了 `ENABLE_OPENWEBUI_TOOLS` 关闭时 `publish_file_from_workspace` 工具丢失的问题。 --- @@ -38,6 +48,18 @@ --- +## 🧩 配套 Files Filter（原始文件必备） + +`GitHub Copilot SDK Files Filter` 是本 Pipe 的配套插件，用于阻止 OpenWebUI 默认 RAG 在 Pipe 接手前抢先处理上传文件。 + +- **作用**: 将上传文件移动到 `copilot_files`，让 Pipe 能直接读取原始二进制。 +- **必要性**: 若未安装，文件可能被提前解析/向量化，Agent 难以拿到原始文件。 +- **v0.1.3 重点**: + - 修复 BYOK 模型 ID 识别（支持 `github_copilot_official_sdk_pipe.xxx` 前缀匹配）。 + - 新增双通道调试日志（`show_debug_log`）：后端 logger + 浏览器控制台。 + +--- + ## ⚙️ 核心配置参数 (Valves) ### 1. 管理员配置 (基础设置) @@ -105,6 +127,15 @@ 2. 创建 **Fine-grained token**，授予 **Account permissions** -> **Copilot Requests** 访问权限。 3. 将生成的 Token 填入插件的 `GH_TOKEN` 配置项中。 +### 3) 认证配置要求（必填） + +你必须至少配置以下一种凭据： + +- `GH_TOKEN`（GitHub Copilot 官方订阅路径），或 +- `BYOK_API_KEY`（OpenAI/Anthropic 自带 Key 路径）。 + +如果两者都未配置，模型列表将不会出现。 + --- ## 📋 常见问题与依赖 (Troubleshooting) diff --git a/docs/plugins/pipes/index.md b/docs/plugins/pipes/index.md index 3630b7d..65d148d 100644 --- a/docs/plugins/pipes/index.md +++ b/docs/plugins/pipes/index.md @@ -15,7 +15,7 @@ Pipes allow you to: ## Available Pipe Plugins -- [GitHub Copilot SDK](github-copilot-sdk.md) (v0.7.0) - Official GitHub Copilot SDK integration. Features **Workspace Isolation**, **Database Persistence**, **Zero-config OpenWebUI Tool Bridge**, **BYOK** support, and **dynamic MCP discovery**. Supports streaming, multimodal, and infinite sessions. [View Deep Dive](github-copilot-sdk-deep-dive.md) | [**View Advanced Tutorial**](github-copilot-sdk-tutorial.md). +- [GitHub Copilot SDK](github-copilot-sdk.md) (v0.8.0) - Official GitHub Copilot SDK integration. Features **Workspace Isolation**, **Database Persistence**, **Zero-config OpenWebUI Tool Bridge**, **BYOK** support, and **dynamic MCP discovery**. Supports streaming, multimodal, and infinite sessions. [View Deep Dive](github-copilot-sdk-deep-dive.md) | [**View Advanced Tutorial**](github-copilot-sdk-tutorial.md). - **[Case Study: GitHub 100 Star Growth Analysis](star-prediction-example.md)** - Learn how to use the GitHub Copilot SDK Pipe with Minimax 2.1 to automatically analyze CSV data and generate project growth reports. - **[Case Study: High-Quality Video to GIF Conversion](video-processing-example.md)** - See how the model uses system-level FFmpeg to accelerate, scale, and optimize colors for screen recordings. diff --git a/docs/plugins/pipes/index.zh.md b/docs/plugins/pipes/index.zh.md index 587ebba..7f57e10 100644 --- a/docs/plugins/pipes/index.zh.md +++ b/docs/plugins/pipes/index.zh.md @@ -15,7 +15,7 @@ Pipes 可以用于： ## 可用的 Pipe 插件 -- [GitHub Copilot SDK](github-copilot-sdk.zh.md) (v0.7.0) - GitHub Copilot SDK 官方集成。具备**工作区安全隔离**、**数据库持久化**、**零配置工具桥接**与**BYOK (自带 Key) 支持**。支持流式输出、打字机思考过程及无限会话。[查看深度架构解析](github-copilot-sdk-deep-dive.zh.md) | [**查看进阶实战教程**](github-copilot-sdk-tutorial.zh.md)。 +- [GitHub Copilot SDK](github-copilot-sdk.zh.md) (v0.8.0) - GitHub Copilot SDK 官方集成。具备**工作区安全隔离**、**数据库持久化**、**零配置工具桥接**与**BYOK (自带 Key) 支持**。支持流式输出、打字机思考过程及无限会话。[查看深度架构解析](github-copilot-sdk-deep-dive.zh.md) | [**查看进阶实战教程**](github-copilot-sdk-tutorial.zh.md)。 - **[实战案例：GitHub 100 Star 增长预测](star-prediction-example.zh.md)** - 展示如何使用 GitHub Copilot SDK Pipe 结合 Minimax 2.1 模型，自动编写脚本分析 CSV 数据并生成详细的项目增长报告。 - **[实战案例：视频高质量 GIF 转换与加速](video-processing-example.zh.md)** - 演示模型如何通过底层 FFmpeg 工具对录屏进行加速、缩放及双阶段色彩优化处理。 diff --git a/plugins/filters/github_copilot_sdk_files_filter/README.md b/plugins/filters/github_copilot_sdk_files_filter/README.md index 0befe50..dacc3d8 100644 --- a/plugins/filters/github_copilot_sdk_files_filter/README.md +++ b/plugins/filters/github_copilot_sdk_files_filter/README.md @@ -1,11 +1,16 @@ # GitHub Copilot SDK Files Filter -**Author:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **Version:** 0.1.2 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT +**Author:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **Version:** 0.1.3 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT This is a dedicated **companion filter plugin** designed specifically for the [GitHub Copilot SDK Pipe](https://openwebui.com/posts/github_copilot_official_sdk_pipe_ce96f7b4). Its core mission is to **protect user-uploaded files from being "pre-processed" by the OpenWebUI core system, ensuring that the Copilot Agent receives the raw files for autonomous analysis.** +## ✨ v0.1.3 Updates (What's New) + +- **🔍 BYOK Model ID Matching Fixed**: Now correctly identifies models in `github_copilot_official_sdk_pipe.xxx` format via prefix matching, in addition to keyword fallback for backward compatibility. (v0.1.3) +- **🐛 Dual-channel Debug Log**: Added `show_debug_log` valve. When enabled, logs are written to both server-side logger and browser console (`console.group`). (v0.1.3) + ## 🎯 Why is this needed? In OpenWebUI's default workflow, when you upload a file (e.g., PDF, Excel, Python script), OpenWebUI automatically initiates a **RAG (Retrieval-Augmented Generation)** process: parsing the file, vectorizing it, extracting text, and injecting it into the prompt. @@ -49,5 +54,5 @@ Default settings work for most users: ## ⚠️ Important Notes -* **Must be used with Copilot SDK Pipe**: If you install this plugin without the main Pipe plugin, uploaded files will simply "disappear" (as no subsequent plugin will look for them). -* **Gemini Filter Compatibility**: Fully compatible with the Gemini Multimodal Filter. Just ensure priorities don't conflict. +- **Must be used with Copilot SDK Pipe**: If you install this plugin without the main Pipe plugin, uploaded files will simply "disappear" (as no subsequent plugin will look for them). +- **Gemini Filter Compatibility**: Fully compatible with the Gemini Multimodal Filter. Just ensure priorities don't conflict. diff --git a/plugins/filters/github_copilot_sdk_files_filter/README_CN.md b/plugins/filters/github_copilot_sdk_files_filter/README_CN.md index 19083d3..bab80e2 100644 --- a/plugins/filters/github_copilot_sdk_files_filter/README_CN.md +++ b/plugins/filters/github_copilot_sdk_files_filter/README_CN.md @@ -1,11 +1,16 @@ # GitHub Copilot SDK 文件过滤器 -**作者:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **版本:** 0.1.2 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT +**作者:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **版本:** 0.1.3 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT 这是一个专门为 [GitHub Copilot SDK Pipe](https://openwebui.com/posts/github_copilot_official_sdk_pipe_ce96f7b4) 设计的**伴侣过滤器插件**。 它的核心使命是：**保护用户上传的文件不被 OpenWebUI 核心系统“抢先处理”，确保 Copilot Agent 能够接收到原始文件并进行自主分析。** +## ✨ 0.1.3 更新内容 (What's New) + +- **🔍 BYOK 模型 ID 匹配修复**: 新增前缀匹配（`github_copilot_official_sdk_pipe.xxx` 格式），修复 BYOK 模型无法被正确识别的问题，关键词兜底保持向后兼容。(v0.1.3) +- **🐛 双通道调试日志**: 新增 `show_debug_log` 配置项，启用后同时向后端日志和浏览器控制台（`console.group`）输出调试信息。(v0.1.3) + ## 🎯 为什么需要它？ 在 OpenWebUI 的默认流程中，当你上传一个文件（如 PDF、Excel、Python 脚本）时，OpenWebUI 会自动启动 **RAG（检索增强生成）** 流程：解析文件、向量化、提取文本并注入到提示词中。 @@ -47,5 +52,5 @@ ## ⚠️ 注意事项 -* **必须配合 Copilot SDK Pipe 使用**：如果你没有安装主 Pipe 插件，本插件将导致上传的文件“凭空消失”。 -* **Gemini Filter 兼容性**：已完美兼容 Gemini 多模态过滤器。只要优先级设置正确，它们可以共存互不干扰。 +- **必须配合 Copilot SDK Pipe 使用**：如果你没有安装主 Pipe 插件，本插件将导致上传的文件“凭空消失”。 +- **Gemini Filter 兼容性**：已完美兼容 Gemini 多模态过滤器。只要优先级设置正确，它们可以共存互不干扰。 diff --git a/plugins/filters/github_copilot_sdk_files_filter/github_copilot_sdk_files_filter.py b/plugins/filters/github_copilot_sdk_files_filter/github_copilot_sdk_files_filter.py index 0cd1780..a163f0e 100644 --- a/plugins/filters/github_copilot_sdk_files_filter/github_copilot_sdk_files_filter.py +++ b/plugins/filters/github_copilot_sdk_files_filter/github_copilot_sdk_files_filter.py @@ -4,13 +4,18 @@ id: github_copilot_sdk_files_filter author: Fu-Jie author_url: https://github.com/Fu-Jie/openwebui-extensions funding_url: https://github.com/open-webui -version: 0.1.2 +version: 0.1.3 openwebui_id: 403a62ee-a596-45e7-be65-fab9cc249dd6 description: A specialized filter to bypass OpenWebUI's default RAG for GitHub Copilot SDK models. It moves uploaded files to a safe location ('copilot_files') so the Copilot Pipe can process them natively without interference. """ from pydantic import BaseModel, Field from typing import Optional, Callable, Awaitable +import logging +import json + + +logger = logging.getLogger(__name__) class Filter: @@ -23,10 +28,62 @@ class Filter: default="copilot_sdk", description="Keyword to identify Copilot models (e.g., 'copilot_sdk').", ) + target_model_prefixes: str = Field( + default="github_copilot_official_sdk_pipe.,github_copilot_sdk_pipe.", + description="Comma-separated model id prefixes to identify Copilot SDK models.", + ) + show_debug_log: bool = Field( + default=False, + description="Whether to print model matching debug logs in backend console.", + ) def __init__(self): self.valves = self.Valves() + def _is_copilot_model(self, model_id: str) -> bool: + if not isinstance(model_id, str) or not model_id: + return False + + current = model_id.strip().lower() + if not current: + return False + + # 1) Prefix match (most reliable for OpenWebUI model id formats) + raw_prefixes = self.valves.target_model_prefixes or "" + prefixes = [p.strip().lower() for p in raw_prefixes.split(",") if p.strip()] + if any(current.startswith(prefix) for prefix in prefixes): + return True + + # 2) Keyword fallback for backward compatibility + keyword = (self.valves.target_model_keyword or "").strip().lower() + return bool(keyword and keyword in current) + + async def _emit_debug_log( + self, + __event_emitter__: Optional[Callable[[dict], Awaitable[None]]], + title: str, + data: dict, + ): + if not self.valves.show_debug_log: + return + + logger.info("[Copilot Files Filter] %s: %s", title, data) + + if not __event_emitter__: + return + + try: + js_code = f""" + (async function() {{ + console.group('🧩 Copilot Files Filter: {title}'); + console.log({json.dumps(data, ensure_ascii=False)}); + console.groupEnd(); + }})(); + """ + await __event_emitter__({"type": "execute", "data": {"code": js_code}}) + except Exception as e: + logger.debug("[Copilot Files Filter] frontend debug emit failed: %s", e) + async def inlet( self, body: dict, @@ -45,8 +102,30 @@ class Filter: current_model = base_model_id if base_model_id else body.get("model", "") + await self._emit_debug_log( + __event_emitter__, + "model-debug", + { + "body_model": body.get("model", ""), + "base_model_id": base_model_id, + "current_model": current_model, + }, + ) + # Check if it's a Copilot model - if self.valves.target_model_keyword.lower() in current_model.lower(): + is_copilot_model = self._is_copilot_model(current_model) + + await self._emit_debug_log( + __event_emitter__, + "match-result", + { + "is_copilot_model": is_copilot_model, + "prefixes": self.valves.target_model_prefixes, + "keyword": self.valves.target_model_keyword, + }, + ) + + if is_copilot_model: # If files exist, move them to 'copilot_files' and clear 'files' # This prevents OpenWebUI from triggering RAG on these files if "files" in body and body["files"]: diff --git a/plugins/pipes/github-copilot-sdk/README.md b/plugins/pipes/github-copilot-sdk/README.md index 60e9548..5b182e1 100644 --- a/plugins/pipes/github-copilot-sdk/README.md +++ b/plugins/pipes/github-copilot-sdk/README.md @@ -1,6 +1,6 @@ # GitHub Copilot SDK Pipe for OpenWebUI -**Author:** [Fu-Jie](https://github.com/Fu-Jie) | **Version:** 0.7.0 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT +**Author:** [Fu-Jie](https://github.com/Fu-Jie) | **Version:** 0.8.0 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT This is an advanced Pipe function for [OpenWebUI](https://github.com/open-webui/open-webui) that integrates the official [GitHub Copilot SDK](https://github.com/github/copilot-sdk). It enables you to use **GitHub Copilot models** (e.g., `gpt-5.2-codex`, `claude-sonnet-4.5`,`gemini-3-pro`, `gpt-5-mini`) **AND** your own models via **BYOK** (OpenAI, Anthropic) directly within OpenWebUI, providing a unified agentic experience with **strict User & Chat-level Workspace Isolation**. @@ -14,13 +14,23 @@ This is an advanced Pipe function for [OpenWebUI](https://github.com/open-webui/ --- -## ✨ v0.7.0 Updates (What's New) +## ✨ v0.8.0 Updates (What's New) -- **🚀 Integrated CLI Management**: The Copilot CLI is now automatically managed and bundled via the `github-copilot-sdk` pip package. No more manual `curl | bash` installation or version mismatches. (v0.7.0) -- **🧠 Native Tool Call UI**: Full adaptation to **OpenWebUI's native tool call UI** and thinking process visualization. (v0.7.0) -- **🏠 OpenWebUI v0.8.0+ Fix**: Resolved "Error getting file content" download failure by switching to absolute path registration for published files. (v0.7.0) -- **🌐 Comprehensive Multi-language Support**: Native localization for status messages in 11 languages (EN, ZH, JA, KO, FR, DE, ES, IT, RU, VI, ID). (v0.7.0) -- **🧹 Architecture Cleanup**: Refactored core setup and optimized reasoning status display for a leaner experience. (v0.7.0) +- **🎛️ Conditional Tool Filtering (P1~P4)**: Four-priority tool permission system. **Default ON**: If no tools are selected in Chat UI (P4), all enabled tools are active. **Whitelist Mode**: Once specific tools are checked, the whitelist strictly filters both OpenWebUI tools and MCP servers. Admin-level `config.enable` (P2) allows global server disabling. (v0.8.0) +- **🔧 File Publish Reliability**: Fixed `Error getting file content` across all storage backends (local/S3/GCS/Azure) by using `Storage.upload_file()` directly in the fallback path. HTML files are no longer blocked by `ALLOWED_FILE_EXTENSIONS` (`?process=false` always applied). (v0.8.0) +- **🌐 HTML Direct Access Link**: When `publish_file_from_workspace` publishes an HTML file, the plugin also provides a directly accessible HTML link for instant in-chat preview/opening. (v0.8.0) +- **🔒 Strict File URL Format**: Published file links must be relative paths starting with `/api/v1/files/` (e.g., `/api/v1/files/{id}/content/html`). Do not use `api/...` and do not prepend any domain. (v0.8.0) +- **🛠️ CLI Built-in Tools Always Available**: `available_tools` is now always `None`, ensuring Copilot CLI built-ins (e.g. `bash`, `create_file`) are never silently blocked regardless of MCP configuration. (v0.8.0) +- **📌 Publish Tool Always Injected**: `publish_file_from_workspace` is no longer lost when `ENABLE_OPENWEBUI_TOOLS` is disabled. (v0.8.0) +- **⚠️ Code Interpreter Limitation**: The `code_interpreter` tool runs in a remote, ephemeral environment. A system prompt warning now clarifies that it cannot access local files or persist changes. (v0.8.0) + +### 🐞 Bug Fixes in v0.8.0 + +- Fixed `{"detail":"[ERROR: Error getting file content]"}` when publishing files under object storage backends by replacing fallback manual copy/DB writes with `Storage.upload_file()`. +- Fixed HTML artifact upload being rejected by `ALLOWED_FILE_EXTENSIONS` by always appending `?process=false` on file upload API calls. +- Fixed invalid artifact links generated as `api/...` or domain-prefixed absolute URLs; links are now constrained to `/api/v1/files/...` relative paths. +- Fixed Copilot CLI built-ins being silently unavailable when no server tools were configured/loaded (which resulted in `available_tools=[]`); now `available_tools` remains `None`. +- Fixed `publish_file_from_workspace` disappearing when `ENABLE_OPENWEBUI_TOOLS` was disabled. --- @@ -33,11 +43,23 @@ This is an advanced Pipe function for [OpenWebUI](https://github.com/open-webui/ - **🧠 Deep Database Integration**: Real-time persistence of TOD·O lists for long-running workflows. - **🌊 Advanced Streaming**: Full support for thinking process/Chain of Thought visualization. - **🖼️ Intelligent Multimodal**: Vision capabilities and raw file analysis support (bypasses RAG for direct binary access). -- **📤 Workspace Artifacts (`publish_file_from_workspace`)**: Agents can generate files (Excel, CSV, HTML reports, etc.) and provide **persistent download links** directly in the chat. +- **📤 Workspace Artifacts (`publish_file_from_workspace`)**: Agents can generate files (Excel, CSV, HTML reports, etc.) and provide **persistent download links** directly in the chat. For HTML files, a direct-access HTML link is also provided. - **🖼️ Interactive Artifacts**: Automatically renders HTML/JS apps generated by the agent directly in the chat interface. --- +## 🧩 Companion Files Filter (Required for raw files) + +`GitHub Copilot SDK Files Filter` is the companion plugin that prevents OpenWebUI's default RAG pre-processing from consuming uploaded files before the Pipe receives them. + +- **What it does**: Moves uploaded files to `copilot_files` so the Pipe can access raw binaries directly. +- **Why it matters**: Without it, uploaded files may be parsed/vectorized early and the Agent may lose direct raw-file access. +- **v0.1.3 highlights**: + - BYOK model-id matching fix (supports `github_copilot_official_sdk_pipe.xxx` prefixes). + - Optional dual-channel debug log (`show_debug_log`) to backend logger + browser console. + +--- + ## ⚙️ Core Configuration (Valves) ### 1. Administrator Settings (Base) @@ -105,6 +127,15 @@ If this plugin has been useful, a **Star** on [OpenWebUI Extensions](https://git 2. Create **Fine-grained token**, granting **Account permissions** -> **Copilot Requests** access. 3. Paste the generated Token into the `GH_TOKEN` field in Valves. +### 3) Authentication Requirement (Mandatory) + +You MUST configure **at least one** credential source: + +- `GH_TOKEN` (GitHub Copilot subscription route), or +- `BYOK_API_KEY` (OpenAI/Anthropic route). + +If neither is configured, the model list will not appear. + --- ## 📋 Troubleshooting & Dependencies diff --git a/plugins/pipes/github-copilot-sdk/README_CN.md b/plugins/pipes/github-copilot-sdk/README_CN.md index d07f135..0721ca9 100644 --- a/plugins/pipes/github-copilot-sdk/README_CN.md +++ b/plugins/pipes/github-copilot-sdk/README_CN.md @@ -1,6 +1,6 @@ # GitHub Copilot SDK 官方管道 -**作者:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **版本:** 0.7.0 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT +**作者:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **版本:** 0.8.0 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT 这是一个用于 [OpenWebUI](https://github.com/open-webui/open-webui) 的高级 Pipe 函数，深度集成了 **GitHub Copilot SDK**。它不仅支持 **GitHub Copilot 官方模型**（如 `gpt-5.2-codex`, `claude-sonnet-4.5`, `gemini-3-pro`, `gpt-5-mini`），还支持 **BYOK (自带 Key)** 模式对接自定义服务商（OpenAI, Anthropic），并具备**严格的用户与会话级工作区隔离**能力，提供统一且安全的 Agent 交互体验。 @@ -14,13 +14,23 @@ --- -## ✨ 0.7.0 更新内容 (What's New) +## ✨ 0.8.0 更新内容 (What's New) -- **🚀 CLI 免维护集成**: Copilot CLI 现在通过 `github-copilot-sdk` pip 包自动同步管理，彻底告别手动 `curl | bash` 安装及版本不匹配问题。(v0.7.0) -- **🧠 原生工具调用 UI**: 全面适配 **OpenWebUI 原生工具调用 UI** 与模型思考过程（思维链）展示。(v0.7.0) -- **🏠 OpenWebUI v0.8.0+ 兼容性修复**: 通过切换为绝对路径注册发布文件，彻底解决了“Error getting file content”无法下载到本地的问题。(v0.7.0) -- **🌐 全面的多语言支持**: 针对状态消息进行了 11 国语言的原生本地化 (中/英/日/韩/法/德/西/意/俄/越/印尼)。(v0.7.0) -- **🧹 架构精简**: 重构了初始化逻辑并优化了推理状态显示，提供更轻量稳健的体验。(v0.7.0) +- **🎛️ 条件工具过滤 (P1~P4)**: 四优先级工具权限体系。**默认全开**: 若未在 Chat UI (P4) 勾选任何工具，则默认启用所有工具；**白名单模式**: 一旦勾选特定工具，即刻进入严格过滤模式，且 MCP server 同步受控；管理员亦可通过 `config.enable` (P2) 全局禁用工具服务器。(v0.8.0) +- **🔧 文件发布全面修复**: 通过在回退路径直接调用 `Storage.upload_file()`，彻底修复了所有存储后端（local/S3/GCS/Azure）下的 `Error getting file content` 问题；同时上传时自动携带 `?process=false`，HTML 文件不再被 `ALLOWED_FILE_EXTENSIONS` 拦截。(v0.8.0) +- **🌐 HTML 直达链接**: 当 `publish_file_from_workspace` 发布的是 HTML 文件时，插件会额外提供可直接访问的 HTML 链接，便于在聊天中即时预览/打开。(v0.8.0) +- **🔒 文件链接格式严格约束**: 发布链接必须是以 `/api/v1/files/` 开头的相对路径（例如 `/api/v1/files/{id}/content/html`）。禁止使用 `api/...`，也禁止拼接任何域名。(v0.8.0) +- **🛠️ CLI 内置工具始终可用**: `available_tools` 统一设为 `None`，Copilot CLI 内置工具（如 `bash`、`create_file`）无论 MCP 配置如何都不会被静默屏蔽。(v0.8.0) +- **📌 发布工具始终注入**: 即使 `ENABLE_OPENWEBUI_TOOLS` 关闭，`publish_file_from_workspace` 工具也不再丢失。(v0.8.0) +- **⚠️ 代码解释器限制**: `code_interpreter` 工具运行在远程临时环境中。系统提示词现已包含警告，明确指出该工具无法访问本地文件或持久化更改。(v0.8.0) + +### 🐞 v0.8.0 Bug 修复说明 + +- 修复了对象存储后端发布文件时出现的 `{"detail":"[ERROR: Error getting file content]"}`，回退路径从手动复制/写库改为 `Storage.upload_file()`。 +- 修复了 HTML 产物被 `ALLOWED_FILE_EXTENSIONS` 拦截的问题，上传接口统一追加 `?process=false`。 +- 修复了产物链接偶发被生成成 `api/...` 或带域名绝对 URL 的问题，现统一限制为 `/api/v1/files/...` 相对路径。 +- 修复了在未配置/未加载任何 server 工具时（最终出现 `available_tools=[]`）Copilot CLI 内置工具被静默禁用的问题，现统一保持 `available_tools=None`。 +- 修复了 `ENABLE_OPENWEBUI_TOOLS` 关闭时 `publish_file_from_workspace` 工具丢失的问题。 --- @@ -33,11 +43,23 @@ - **🧠 深度数据库集成**: 实时持久化 TOD·O 列表到 UI 进度条。 - **🌊 深度推理展示**: 完整支持模型思考过程 (Thinking Process) 的流式渲染。 - **🖼️ 智能多模态**: 完整支持图像识别与附件上传分析（绕过 RAG 直接访问原始二进制内容）。 -- **📤 工作区产物工具 (`publish_file_from_workspace`)**: Agent 可生成文件（Excel、CSV、HTML 报告等）并直接在聊天中提供**持久化下载链接**。 +- **📤 工作区产物工具 (`publish_file_from_workspace`)**: Agent 可生成文件（Excel、CSV、HTML 报告等）并直接在聊天中提供**持久化下载链接**。若为 HTML 文件，还会额外提供可直接访问的 HTML 链接。 - **🖼️ 交互式伪影 (Artifacts)**: 自动渲染 Agent 生成的 HTML/JS 应用程序，直接在聊天界面交互。 --- +## 🧩 配套 Files Filter（原始文件必备） + +`GitHub Copilot SDK Files Filter` 是本 Pipe 的配套插件，用于阻止 OpenWebUI 默认 RAG 在 Pipe 接手前抢先处理上传文件。 + +- **作用**: 将上传文件移动到 `copilot_files`，让 Pipe 能直接读取原始二进制。 +- **必要性**: 若未安装，文件可能被提前解析/向量化，Agent 难以拿到原始文件。 +- **v0.1.3 重点**: + - 修复 BYOK 模型 ID 识别（支持 `github_copilot_official_sdk_pipe.xxx` 前缀匹配）。 + - 新增双通道调试日志（`show_debug_log`）：后端 logger + 浏览器控制台。 + +--- + ## ⚙️ 核心配置参数 (Valves) ### 1. 管理员配置 (基础设置) @@ -105,7 +127,16 @@ 2. 创建 **Fine-grained token**，授予 **Account permissions** -> **Copilot Requests** 访问权限。 3. 将生成的 Token 填入插件的 `GH_TOKEN` 配置项中。 -### 3) 配套插件 (强烈推荐) +### 3) 认证配置要求（必填） + +你必须至少配置以下一种凭据： + +- `GH_TOKEN`（GitHub Copilot 官方订阅路径），或 +- `BYOK_API_KEY`（OpenAI/Anthropic 自带 Key 路径）。 + +如果两者都未配置，模型列表将不会出现。 + +### 4) 配套插件 (强烈推荐) 为了获得最佳的文件处理体验，请安装 [GitHub Copilot SDK Files Filter](https://openwebui.com/posts/403a62ee-a596-45e7-be65-fab9cc249dd6)。 diff --git a/plugins/pipes/github-copilot-sdk/github_copilot_sdk.py b/plugins/pipes/github-copilot-sdk/github_copilot_sdk.py index b2f4650..29eea52 100644 --- a/plugins/pipes/github-copilot-sdk/github_copilot_sdk.py +++ b/plugins/pipes/github-copilot-sdk/github_copilot_sdk.py @@ -5,7 +5,7 @@ author_url: https://github.com/Fu-Jie/openwebui-extensions funding_url: https://github.com/open-webui openwebui_id: ce96f7b4-12fc-4ac3-9a01-875713e69359 description: Integrate GitHub Copilot SDK. Supports dynamic models, multi-turn conversation, streaming, multimodal input, infinite sessions, and frontend debug logging. -version: 0.7.0 +version: 0.8.0 requirements: github-copilot-sdk==0.1.25 """ @@ -17,11 +17,9 @@ import tempfile import asyncio import logging import shutil -import subprocess import hashlib import aiohttp import contextlib -import traceback from pathlib import Path from typing import Optional, Union, AsyncGenerator, List, Any, Dict, Literal, Tuple from types import SimpleNamespace @@ -46,6 +44,7 @@ from open_webui.models.tools import Tools from open_webui.models.users import Users from open_webui.models.files import Files, FileForm from open_webui.config import UPLOAD_DIR, DATA_DIR +from open_webui.storage.provider import Storage import mimetypes import uuid import shutil @@ -138,7 +137,8 @@ BASE_GUIDELINES = ( " - **Philosophy**: Visual Artifacts (HTML/Mermaid) and Downloadable Files are **COMPLEMENTARY**. Always aim to provide BOTH: instant visual insight in the chat AND a persistent file for the user to keep.\n" " - **The Rule**: When the user needs to *possess* data (download/export), you MUST publish it. Creating a local file alone is useless because the user cannot access your container.\n" " - **Implicit Requests**: If asked to 'export', 'get link', or 'save', automatically trigger this sequence.\n" - " - **Execution Sequence**: 1. **Write Local**: Create file in `.` (current directory). 2. **Publish**: Call `publish_file_from_workspace(filename='your_file.ext')`. 3. **Link**: Present the `download_url` as a Markdown link.\n" + " - **Execution Sequence**: 1. **Write Local**: Create file in `.` (current directory). 2. **Publish**: Call `publish_file_from_workspace(filename='your_file.ext')`. 3. **Link**: Present the result based on file type. **For HTML files**: the tool returns both `download_url` (raw file) and `view_url` (`/content/html` format). You MUST present `view_url` as a **[Preview]** link that opens directly in the browser, AND `download_url` as a **[Download]** link. **For image files** (.png, .jpg, .gif, .svg, etc.): embed directly using `![caption](download_url)` — NEVER use a text link for images. For all other files: present `download_url` as a single download link.\n" + " - **URL Format is STRICT**: File links MUST be relative paths starting with `/api/v1/files/` (for example: `/api/v1/files/{id}/content` or `/api/v1/files/{id}/content/html`). NEVER output `api/...` (missing leading slash). NEVER prepend any domain (such as `https://example.com/...`) even if a domain appears in conversation context.\n" " - **Bypass RAG**: This protocol automatically handles S3 storage and bypasses RAG, ensuring 100% accurate data delivery.\n" "6. **TODO Visibility**: Every time you call the `update_todo` tool, you **MUST** immediately follow up with a beautifully formatted **Markdown summary** of the current TODO list. Use task checkboxes (`- [ ]`), progress indicators, and clear headings so the user can see the status directly in the chat.\n" "7. **Python Execution Standard**: For ANY task requiring Python logic (not just data analysis), you **MUST NOT** embed multi-line code directly in a shell command (e.g., using `python -c` or `<< 'EOF'`).\n" @@ -197,10 +197,6 @@ class Pipe: default=True, description="Enable Direct MCP Client connection (Recommended).", ) - ENABLE_TOOL_CACHE: bool = Field( - default=True, - description="Cache OpenWebUI tools and MCP servers (performance optimization).", - ) REASONING_EFFORT: Literal["low", "medium", "high", "xhigh"] = Field( default="medium", description="Reasoning effort level (low, medium, high). Only affects standard Copilot models (not BYOK).", @@ -251,6 +247,10 @@ class Pipe: default="/app/backend/data/uploads", description="Path to OpenWebUI uploads directory (for file processing).", ) + MODEL_CACHE_TTL: int = Field( + default=3600, + description="Model list cache TTL in seconds. Set to 0 to disable cache (always fetch). Default: 3600 (1 hour).", + ) BYOK_TYPE: Literal["openai", "anthropic"] = Field( default="openai", @@ -314,10 +314,6 @@ class Pipe: default=True, description="Enable dynamic MCP server loading (overrides global).", ) - ENABLE_TOOL_CACHE: bool = Field( - default=True, - description="Enable Tool/MCP configuration caching for this user.", - ) # BYOK User Overrides BYOK_API_KEY: str = Field( @@ -352,8 +348,7 @@ class Pipe: _model_cache: List[dict] = [] # Model list cache _standard_model_ids: set = set() # Track standard model IDs _last_byok_config_hash: str = "" # Track BYOK config for cache invalidation - _tool_cache = None # Cache for converted OpenWebUI tools - _mcp_server_cache = None # Cache for MCP server config + _last_model_cache_time: float = 0 # Timestamp of last model cache refresh _env_setup_done = False # Track if env setup has been completed _last_update_check = 0 # Timestamp of last CLI update check @@ -717,46 +712,32 @@ class Pipe: uv = self._get_user_valves(__user__) enable_tools = uv.ENABLE_OPENWEBUI_TOOLS enable_openapi = uv.ENABLE_OPENAPI_SERVER - enable_cache = uv.ENABLE_TOOL_CACHE - # 2. If all tool types are disabled, return empty immediately + # 2. Publish tool is always injected, regardless of other settings + chat_ctx = self._get_chat_context(body, __metadata__) + chat_id = chat_ctx.get("chat_id") + file_tool = self._get_publish_file_tool(__user__, chat_id, __request__) + final_tools = [file_tool] if file_tool else [] + + # 3. If all OpenWebUI tool types are disabled, skip loading and return early if not enable_tools and not enable_openapi: - return [] + return final_tools - # 3. Check Cache - if enable_cache and self._tool_cache is not None: - await self._emit_debug_log( - "ℹ️ Using cached OpenWebUI tools.", __event_call__ - ) - # Create a shallow copy to append user-specific tools without polluting cache - tools = list(self._tool_cache) + # 4. Extract chat-level tool selection (P4: user selection from Chat UI) + chat_tool_ids = None + if __metadata__ and isinstance(__metadata__, dict): + chat_tool_ids = __metadata__.get("tool_ids") or None - # Inject File Publish Tool - chat_ctx = self._get_chat_context(body, __metadata__) - chat_id = chat_ctx.get("chat_id") - file_tool = self._get_publish_file_tool(__user__, chat_id, __request__) - if file_tool: - tools.append(file_tool) - - return tools - - # Load OpenWebUI tools dynamically + # 5. Load OpenWebUI tools dynamically (always fresh, no cache) openwebui_tools = await self._load_openwebui_tools( body=body, __user__=__user__, __event_call__=__event_call__, enable_tools=enable_tools, enable_openapi=enable_openapi, + chat_tool_ids=chat_tool_ids, ) - # Update Cache - if enable_cache: - self._tool_cache = openwebui_tools - await self._emit_debug_log( - "✅ OpenWebUI tools cached for subsequent requests.", __event_call__ - ) - - # Log details only when cache is cold if openwebui_tools: tool_names = [t.name for t in openwebui_tools] await self._emit_debug_log( @@ -770,15 +751,7 @@ class Pipe: __event_call__, ) - # Create a shallow copy to append user-specific tools without polluting cache - final_tools = list(openwebui_tools) - - # Inject File Publish Tool - chat_ctx = self._get_chat_context(body, __metadata__) - chat_id = chat_ctx.get("chat_id") - file_tool = self._get_publish_file_tool(__user__, chat_id, __request__) - if file_tool: - final_tools.append(file_tool) + final_tools.extend(openwebui_tools) return final_tools @@ -892,7 +865,9 @@ class Pipe: import aiohttp base_url = str(__request__.base_url).rstrip("/") - upload_url = f"{base_url}/api/v1/files/" + # ?process=false skips RAG processing AND the + # ALLOWED_FILE_EXTENSIONS restriction (which blocks html/htm) + upload_url = f"{base_url}/api/v1/files/?process=false" async with aiohttp.ClientSession() as session: with open(target_path, "rb") as f: @@ -925,14 +900,25 @@ class Pipe: except Exception as e: logger.error(f"API upload failed: {e}") - # 4. Fallback: Manual DB Insert (Local only) + # 4. Fallback: Use Storage.upload_file directly (S3/Local/GCS/Azure compatible) if not api_success: file_id = str(uuid.uuid4()) safe_filename = target_path.name - dest_path = Path(UPLOAD_DIR) / f"{file_id}_{safe_filename}" - await asyncio.to_thread(shutil.copy2, target_path, dest_path) + storage_filename = f"{file_id}_{safe_filename}" - db_path = str(dest_path) + def _upload_via_storage(): + with open(target_path, "rb") as f: + _, stored_path = Storage.upload_file( + f, + storage_filename, + { + "OpenWebUI-User-Id": user_id, + "OpenWebUI-File-Id": file_id, + }, + ) + return stored_path + + db_path = await asyncio.to_thread(_upload_via_storage) file_form = FileForm( id=file_id, @@ -943,7 +929,7 @@ class Pipe: "name": safe_filename, "content_type": mimetypes.guess_type(safe_filename)[0] or "text/plain", - "size": os.path.getsize(dest_path), + "size": os.path.getsize(target_path), "source": "copilot_workspace_publish", "skip_rag": True, }, @@ -952,16 +938,16 @@ class Pipe: # 5. Result download_url = f"/api/v1/files/{file_id}/content" - view_url = download_url - is_html = safe_filename.lower().endswith(".html") + is_html = safe_filename.lower().endswith((".html", ".htm")) - # For HTML files, if user is admin, provide a direct view link (/content/html) - if is_html and is_admin: + # For HTML files, provide a direct view link (/content/html) for browser preview + view_url = None + if is_html: view_url = f"{download_url}/html" # Localized output msg = self._get_translation(user_lang, "publish_success") - if is_html and is_admin: + if is_html: hint = self._get_translation( user_lang, "publish_hint_html", @@ -977,13 +963,16 @@ class Pipe: download_url=download_url, ) - return { + result = { "file_id": file_id, "filename": safe_filename, "download_url": download_url, "message": msg, "hint": hint, } + if is_html and view_url: + result["view_url"] = view_url + return result except Exception as e: return {"error": str(e)} @@ -1206,6 +1195,31 @@ class Pipe: params_type=ParamsModel, )(_tool) + def _read_tool_server_connections(self) -> list: + """ + Read tool server connections directly from the database to avoid stale + in-memory state in multi-worker deployments. + Falls back to the in-memory PersistentConfig value if DB read fails. + """ + try: + from open_webui.config import get_config + + config_data = get_config() + connections = config_data.get("tool_server", {}).get("connections", None) + if connections is not None: + return connections if isinstance(connections, list) else [] + except Exception as e: + logger.debug( + f"[Tools] DB config read failed, using in-memory fallback: {e}" + ) + + # Fallback: in-memory value (may be stale in multi-worker) + if hasattr(TOOL_SERVER_CONNECTIONS, "value") and isinstance( + TOOL_SERVER_CONNECTIONS.value, list + ): + return TOOL_SERVER_CONNECTIONS.value + return [] + def _build_openwebui_request(self, user: dict = None, token: str = None): """Build a more complete request-like object with dynamically loaded OpenWebUI configs.""" # Dynamically build config from the official registry @@ -1219,12 +1233,9 @@ class Pipe: setattr(config, item.env_name, val) # Critical Fix: Explicitly sync TOOL_SERVER_CONNECTIONS to ensure OpenAPI tools work - # PERSISTENT_CONFIG_REGISTRY might not contain TOOL_SERVER_CONNECTIONS in all versions - if not hasattr(config, "TOOL_SERVER_CONNECTIONS"): - if hasattr(TOOL_SERVER_CONNECTIONS, "value"): - config.TOOL_SERVER_CONNECTIONS = TOOL_SERVER_CONNECTIONS.value - else: - config.TOOL_SERVER_CONNECTIONS = TOOL_SERVER_CONNECTIONS + # Read directly from DB to avoid stale in-memory state in multi-worker deployments + fresh_connections = self._read_tool_server_connections() + config.TOOL_SERVER_CONNECTIONS = fresh_connections app_state = SimpleNamespace( config=config, @@ -1282,6 +1293,7 @@ class Pipe: __event_call__=None, enable_tools: bool = True, enable_openapi: bool = True, + chat_tool_ids: Optional[list] = None, ): """Load OpenWebUI tools and convert them to Copilot SDK tools.""" if isinstance(__user__, (list, tuple)): @@ -1300,17 +1312,20 @@ class Pipe: # --- PROBE LOG --- if __event_call__: - conn_status = "Missing" - if hasattr(TOOL_SERVER_CONNECTIONS, "value"): - val = TOOL_SERVER_CONNECTIONS.value - conn_status = ( - f"List({len(val)})" if isinstance(val, list) else str(type(val)) - ) + conn_list = self._read_tool_server_connections() + conn_summary = [] + for i, s in enumerate(conn_list): + if isinstance(s, dict): + s_id = s.get("info", {}).get("id") or s.get("id") or str(i) + s_type = s.get("type", "openapi") + s_enabled = s.get("config", {}).get("enable", False) + conn_summary.append( + {"id": s_id, "type": s_type, "enable": s_enabled} + ) await self._emit_debug_log( - f"[Tools Debug] Entry. UserID: {user_id}, EnableTools: {enable_tools}, EnableOpenAPI: {enable_openapi}, Connections: {conn_status}", + f"[Tools] TOOL_SERVER_CONNECTIONS ({len(conn_summary)} entries): {conn_summary}", __event_call__, - debug_enabled=True, ) # ----------------- @@ -1327,67 +1342,85 @@ class Pipe: # 2. Get OpenAPI Tool Server tools if enable_openapi: - if hasattr(TOOL_SERVER_CONNECTIONS, "value"): - raw_connections = TOOL_SERVER_CONNECTIONS.value + raw_connections = self._read_tool_server_connections() - # Handle Pydantic model vs List vs Dict - connections = [] - if isinstance(raw_connections, list): - connections = raw_connections - elif hasattr(raw_connections, "dict"): - connections = raw_connections.dict() - elif hasattr(raw_connections, "model_dump"): - connections = raw_connections.model_dump() + # Handle Pydantic model vs List vs Dict + connections = [] + if isinstance(raw_connections, list): + connections = raw_connections + elif hasattr(raw_connections, "dict"): + connections = raw_connections.dict() + elif hasattr(raw_connections, "model_dump"): + connections = raw_connections.model_dump() + + # Debug logging for connections + if self.valves.DEBUG: + await self._emit_debug_log( + f"[Tools] Found {len(connections)} server connections (Type: {type(raw_connections)})", + __event_call__, + ) + + for idx, server in enumerate(connections): + # Handle server item type + s_type = ( + server.get("type", "openapi") + if isinstance(server, dict) + else getattr(server, "type", "openapi") + ) + + # P2: config.enable check — skip admin-disabled servers + s_config = ( + server.get("config", {}) + if isinstance(server, dict) + else getattr(server, "config", {}) + ) + s_enabled = ( + s_config.get("enable", False) + if isinstance(s_config, dict) + else getattr(s_config, "enable", False) + ) + if not s_enabled: + if self.valves.DEBUG: + await self._emit_debug_log( + f"[Tools] Skipped disabled server at index {idx}", + __event_call__, + ) + continue + + # Handle server ID: Priority info.id > server.id > index + s_id = None + if isinstance(server, dict): + info = server.get("info", {}) + s_id = info.get("id") or server.get("id") + else: + info = getattr(server, "info", {}) + if isinstance(info, dict): + s_id = info.get("id") + else: + s_id = getattr(info, "id", None) + + if not s_id: + s_id = getattr(server, "id", None) + + if not s_id: + s_id = str(idx) - # Debug logging for connections if self.valves.DEBUG: await self._emit_debug_log( - f"[Tools] Found {len(connections)} server connections (Type: {type(raw_connections)})", + f"[Tools] Checking Server: ID={s_id}, Type={s_type}", __event_call__, ) - for idx, server in enumerate(connections): - # Handle server item type - s_type = ( - server.get("type", "openapi") - if isinstance(server, dict) - else getattr(server, "type", "openapi") + if s_type == "openapi": + # Ensure we don't add empty IDs, though fallback to idx should prevent this + if s_id: + tool_ids.append(f"server:{s_id}") + elif self.valves.DEBUG: + await self._emit_debug_log( + f"[Tools] Skipped non-OpenAPI server: {s_id} ({s_type})", + __event_call__, ) - # Handle server ID: Priority info.id > server.id > index - s_id = None - if isinstance(server, dict): - info = server.get("info", {}) - s_id = info.get("id") or server.get("id") - else: - info = getattr(server, "info", {}) - if isinstance(info, dict): - s_id = info.get("id") - else: - s_id = getattr(info, "id", None) - - if not s_id: - s_id = getattr(server, "id", None) - - if not s_id: - s_id = str(idx) - - if self.valves.DEBUG: - await self._emit_debug_log( - f"[Tools] Checking Server: ID={s_id}, Type={s_type}", - __event_call__, - ) - - if s_type == "openapi": - # Ensure we don't add empty IDs, though fallback to idx should prevent this - if s_id: - tool_ids.append(f"server:{s_id}") - elif self.valves.DEBUG: - await self._emit_debug_log( - f"[Tools] Skipped non-OpenAPI server: {s_id} ({s_type})", - __event_call__, - ) - if ( not tool_ids and not enable_tools ): # No IDs and no built-ins either if tools disabled @@ -1397,6 +1430,16 @@ class Pipe: ) return [] + # P4: Chat tool_ids whitelist — only active when user explicitly selected tools + if chat_tool_ids: + chat_tool_ids_set = set(chat_tool_ids) + filtered = [tid for tid in tool_ids if tid in chat_tool_ids_set] + await self._emit_debug_log( + f"[Tools] tool_ids whitelist active: {len(tool_ids)} → {len(filtered)} (selected: {chat_tool_ids})", + __event_call__, + ) + tool_ids = filtered + if self.valves.DEBUG and tool_ids: await self._emit_debug_log( f"[Tools] Requesting tool IDs: {tool_ids}", __event_call__ @@ -1469,7 +1512,28 @@ class Pipe: # Fetch Built-in Tools (Web Search, Memory, etc.) if enable_tools: try: + # Resolve real model dict from DB to respect meta.builtinTools config + model_dict = {} + model_id = body.get("model", "") if isinstance(body, dict) else "" + if model_id: + try: + from open_webui.models.models import Models as _Models + + model_record = _Models.get_model_by_id(model_id) + if model_record: + model_dict = {"info": model_record.model_dump()} + except Exception: + pass + # Get builtin tools + # Open all feature gates so filtering is driven solely by + # model.meta.builtinTools (defaults to all-enabled when absent). + all_features = { + "memory": True, + "web_search": True, + "image_generation": True, + "code_interpreter": True, + } builtin_tools = get_builtin_tools( self._build_openwebui_request(user_data), { @@ -1477,16 +1541,8 @@ class Pipe: "__chat_id__": extra_params.get("__chat_id__"), "__message_id__": extra_params.get("__message_id__"), }, - model={ - "info": { - "meta": { - "capabilities": { - "web_search": True, - "image_generation": True, - } - } - } - }, # Mock capabilities to allow all globally enabled tools + features=all_features, + model=model_dict, # model.meta.builtinTools controls which categories are active ) if builtin_tools: tools_dict.update(builtin_tools) @@ -1504,16 +1560,15 @@ class Pipe: tool_metadata_cache = {} server_metadata_cache = {} - # Pre-build server metadata cache from TOOL_SERVER_CONNECTIONS - if hasattr(TOOL_SERVER_CONNECTIONS, "value"): - for server in TOOL_SERVER_CONNECTIONS.value: - server_id = server.get("id") or server.get("info", {}).get("id") - if server_id: - info = server.get("info", {}) - server_metadata_cache[server_id] = { - "name": info.get("name") or server_id, - "description": info.get("description", ""), - } + # Pre-build server metadata cache from DB-fresh tool server connections + for server in self._read_tool_server_connections(): + server_id = server.get("id") or server.get("info", {}).get("id") + if server_id: + info = server.get("info", {}) + server_metadata_cache[server_id] = { + "name": info.get("name") or server_id, + "description": info.get("description", ""), + } for tool_name, tool_def in tools_dict.items(): tool_id = tool_def.get("tool_id", "") @@ -1578,7 +1633,10 @@ class Pipe: return converted_tools def _parse_mcp_servers( - self, __event_call__=None, enable_mcp: bool = True, enable_cache: bool = True + self, + __event_call__=None, + enable_mcp: bool = True, + chat_tool_ids: Optional[list] = None, ) -> Optional[dict]: """ Dynamically load MCP servers from OpenWebUI TOOL_SERVER_CONNECTIONS. @@ -1587,17 +1645,22 @@ class Pipe: if not enable_mcp: return None - # Check Cache - if enable_cache and self._mcp_server_cache is not None: - return self._mcp_server_cache - mcp_servers = {} - # Iterate over OpenWebUI Tool Server Connections - if hasattr(TOOL_SERVER_CONNECTIONS, "value"): - connections = TOOL_SERVER_CONNECTIONS.value - else: - connections = [] + # Read MCP servers directly from DB to avoid stale in-memory cache + connections = self._read_tool_server_connections() + + if __event_call__: + mcp_summary = [] + for s in connections if isinstance(connections, list) else []: + if isinstance(s, dict) and s.get("type") == "mcp": + s_id = s.get("info", {}).get("id") or s.get("id", "?") + s_enabled = s.get("config", {}).get("enable", False) + mcp_summary.append({"id": s_id, "enable": s_enabled}) + self._emit_debug_log_sync( + f"[MCP] TOOL_SERVER_CONNECTIONS MCP entries ({len(mcp_summary)}): {mcp_summary}", + __event_call__, + ) for conn in connections: if conn.get("type") == "mcp": @@ -1605,6 +1668,18 @@ class Pipe: # Use ID from info or generate one raw_id = info.get("id", f"mcp-server-{len(mcp_servers)}") + # P2: config.enable check — skip admin-disabled servers + mcp_config = conn.get("config", {}) + if not mcp_config.get("enable", False): + self._emit_debug_log_sync( + f"[MCP] Skipped disabled server: {raw_id}", __event_call__ + ) + continue + + # P4: chat_tool_ids whitelist — if user selected tools, only include matching servers + if chat_tool_ids and f"server:{raw_id}" not in chat_tool_ids: + continue + # Sanitize server_id (using same logic as tools) server_id = re.sub(r"[^a-zA-Z0-9_-]", "_", raw_id) if not server_id or re.match(r"^[_.-]+$", server_id): @@ -1636,7 +1711,6 @@ class Pipe: headers.update(custom_headers) # Get filtering configuration - mcp_config = conn.get("config", {}) function_filter = mcp_config.get("function_name_filter_list", "") allowed_tools = ["*"] @@ -1658,10 +1732,6 @@ class Pipe: f"🔌 MCP Integrated: {server_id}", __event_call__ ) - # Update Cache - if self.valves.ENABLE_TOOL_CACHE: - self._mcp_server_cache = mcp_servers - return mcp_servers if mcp_servers else None async def _emit_debug_log( @@ -1899,6 +1969,22 @@ class Pipe: ) break + # Append Code Interpreter Warning + code_interpreter_warning = ( + "\n\n[System Note]\n" + "The `execute_code` tool (builtin category: `code_interpreter`) executes code in a remote, ephemeral environment. " + "It cannot access files in your local workspace or persist changes. " + "Use it only for calculation or logic verification, not for file manipulation." + "\n" + "For links returned by `publish_file_from_workspace`, URL formatting is strict: " + "always use relative paths that start with `/api/v1/files/`. " + "Do not output `api/...` and do not prepend any domain." + ) + if system_prompt_content: + system_prompt_content += code_interpreter_warning + else: + system_prompt_content = code_interpreter_warning.strip() + return system_prompt_content, system_prompt_source def _get_workspace_dir(self, user_id: str = None, chat_id: str = None) -> str: @@ -1969,7 +2055,7 @@ class Pipe: is_admin: bool = False, user_id: str = None, enable_mcp: bool = True, - enable_cache: bool = True, + chat_tool_ids: Optional[list] = None, __event_call__=None, ): """Build SessionConfig for Copilot SDK.""" @@ -2018,7 +2104,7 @@ class Pipe: } mcp_servers = self._parse_mcp_servers( - __event_call__, enable_mcp=enable_mcp, enable_cache=enable_cache + __event_call__, enable_mcp=enable_mcp, chat_tool_ids=chat_tool_ids ) # Prepare session config parameters @@ -2061,12 +2147,12 @@ class Pipe: if mcp_servers: session_params["mcp_servers"] = mcp_servers - # Critical Fix: When using MCP, available_tools must be None to allow dynamic discovery - session_params["available_tools"] = None - else: - session_params["available_tools"] = ( - [t.name for t in custom_tools] if custom_tools else None - ) + + # Always set available_tools=None so the Copilot CLI's built-in tools + # (e.g. bash, create_file) remain accessible alongside our custom tools. + # Custom tools are registered via the 'tools' param; whitelist filtering + # via available_tools would silently block CLI built-ins. + session_params["available_tools"] = None if provider_config: session_params["provider"] = provider_config @@ -2338,7 +2424,7 @@ class Pipe: # 1. Environment Setup (Only if needed or not done) if needs_setup: - self._setup_env(token=token, skip_cli_install=True) + self._setup_env(token=token) self.__class__._last_update_check = now else: # Still inject token for BYOK real-time updates @@ -2380,6 +2466,19 @@ class Pipe: current_config_str = f"{token}|{uv.BYOK_BASE_URL or self.valves.BYOK_BASE_URL}|{uv.BYOK_API_KEY or self.valves.BYOK_API_KEY}|{self.valves.BYOK_BEARER_TOKEN}" current_config_hash = hashlib.md5(current_config_str.encode()).hexdigest() + # TTL-based cache expiry + cache_ttl = self.valves.MODEL_CACHE_TTL + if ( + self._model_cache + and cache_ttl > 0 + and (now - self.__class__._last_model_cache_time) > cache_ttl + ): + if self.valves.DEBUG: + logger.info( + f"[Pipes] Model cache expired (TTL={cache_ttl}s). Invalidating." + ) + self.__class__._model_cache = [] + if ( self._model_cache and self.__class__._last_byok_config_hash != current_config_hash @@ -2397,8 +2496,12 @@ class Pipe: if self.valves.DEBUG: logger.info("[Pipes] Refreshing model cache...") try: - # Use effective token for fetching - self._setup_env(token=token, skip_cli_install=True) + # Use effective token for fetching. + # If COPILOT_CLI_PATH is missing (e.g. env cleared after worker restart), + # force a full re-discovery by resetting _env_setup_done first. + if not os.environ.get("COPILOT_CLI_PATH"): + self.__class__._env_setup_done = False + self._setup_env(token=token) # Fetch BYOK models if configured byok = [] @@ -2478,7 +2581,24 @@ class Pipe: ) self._model_cache = standard + byok + self.__class__._last_model_cache_time = now if not self._model_cache: + has_byok = bool( + (uv.BYOK_BASE_URL or self.valves.BYOK_BASE_URL) + and ( + uv.BYOK_API_KEY + or self.valves.BYOK_API_KEY + or uv.BYOK_BEARER_TOKEN + or self.valves.BYOK_BEARER_TOKEN + ) + ) + if not token and not has_byok: + return [ + { + "id": "no_token", + "name": "⚠️ No credentials configured. Please set GH_TOKEN or BYOK settings in Valves.", + } + ] return [ { "id": "warming_up", @@ -2530,8 +2650,6 @@ class Pipe: debug_enabled: bool = False, token: str = None, enable_mcp: bool = True, - enable_cache: bool = True, - skip_cli_install: bool = False, # Kept for call-site compatibility, no longer used __event_emitter__=None, user_lang: str = "en-US", ): @@ -2548,7 +2666,6 @@ class Pipe: __event_call__, debug_enabled, enable_mcp=enable_mcp, - enable_cache=enable_cache, ) return @@ -2762,7 +2879,6 @@ class Pipe: __event_call__=None, debug_enabled: bool = False, enable_mcp: bool = True, - enable_cache: bool = True, ): """Sync MCP configuration to ~/.copilot/config.json.""" path = os.path.expanduser("~/.copilot/config.json") @@ -2786,9 +2902,7 @@ class Pipe: pass return - mcp = self._parse_mcp_servers( - __event_call__, enable_mcp=enable_mcp, enable_cache=enable_cache - ) + mcp = self._parse_mcp_servers(__event_call__, enable_mcp=enable_mcp) if not mcp: return try: @@ -2866,9 +2980,13 @@ class Pipe: ) chat_id = chat_ctx.get("chat_id") or "default" - # Determine effective MCP and cache settings + # Determine effective MCP settings effective_mcp = user_valves.ENABLE_MCP_SERVER - effective_cache = user_valves.ENABLE_TOOL_CACHE + + # P4: Chat tool_ids whitelist — extract once, reuse for both OpenAPI and MCP + chat_tool_ids = None + if __metadata__ and isinstance(__metadata__, dict): + chat_tool_ids = __metadata__.get("tool_ids") or None user_ctx = await self._get_user_context(__user__, __event_call__, __request__) user_lang = user_ctx["user_language"] @@ -2879,7 +2997,6 @@ class Pipe: debug_enabled=effective_debug, token=effective_token, enable_mcp=effective_mcp, - enable_cache=effective_cache, __event_emitter__=__event_emitter__, user_lang=user_lang, ) @@ -3125,7 +3242,7 @@ class Pipe: # Check MCP Servers mcp_servers = self._parse_mcp_servers( - __event_call__, enable_mcp=effective_mcp, enable_cache=effective_cache + __event_call__, enable_mcp=effective_mcp, chat_tool_ids=chat_tool_ids ) mcp_server_names = list(mcp_servers.keys()) if mcp_servers else [] if mcp_server_names: @@ -3180,15 +3297,13 @@ class Pipe: mcp_servers = self._parse_mcp_servers( __event_call__, enable_mcp=effective_mcp, - enable_cache=effective_cache, + chat_tool_ids=chat_tool_ids, ) if mcp_servers: resume_params["mcp_servers"] = mcp_servers - resume_params["available_tools"] = None - else: - resume_params["available_tools"] = ( - [t.name for t in custom_tools] if custom_tools else None - ) + + # Always None: let CLI built-ins (bash etc.) remain available. + resume_params["available_tools"] = None # Always inject the latest system prompt in 'replace' mode # This handles both custom models and user-defined system messages @@ -3274,7 +3389,7 @@ class Pipe: is_admin=is_admin, user_id=user_id, enable_mcp=effective_mcp, - enable_cache=effective_cache, + chat_tool_ids=chat_tool_ids, __event_call__=__event_call__, ) diff --git a/plugins/pipes/github-copilot-sdk/v0.8.0.md b/plugins/pipes/github-copilot-sdk/v0.8.0.md new file mode 100644 index 0000000..e46b4e5 --- /dev/null +++ b/plugins/pipes/github-copilot-sdk/v0.8.0.md @@ -0,0 +1,127 @@ +# 🚀 GitHub Copilot SDK Pipe v0.8.0: Conditional Tool Filtering & Publish Reliability 🎛️ + +**GitHub Copilot SDK Pipe v0.8.0** — A major control and reliability upgrade. This release introduces a four-priority tool permission system that makes tool access fully configurable per conversation, fixes file publishing across all storage backends, and ensures CLI built-in tools are never accidentally silenced. + +--- + +## 📦 Quick Installation + +- **GitHub Copilot SDK (Pipe)**: [Install v0.8.0](https://openwebui.com/posts/ce96f7b4-12fc-4ac3-9a01-875713e69359) +- **GitHub Copilot SDK (Filter)**: [Install v0.1.3](https://openwebui.com/posts/403a62ee-a596-45e7-be65-fab9cc249dd6) + +--- + +## 🚀 What's New in v0.8.0 + +### 1. Conditional Tool Filtering (P1~P4) — The Headline Feature + +This release introduces a **four-priority tool permission system** that gives you precise control over which tools are active in each conversation. + +| Priority | Scope | Mechanism | +| :--- | :--- | :--- | +| **P1** | Global | Pipe-level Valve switches (`ENABLE_OPENWEBUI_TOOLS`, `ENABLE_OPENAPI_SERVER`, `ENABLE_MCP`) | +| **P2** | Server-level | Admin `config.enable` — disable a tool server from the OpenWebUI admin panel without changing code | +| **P3** | User-level | Per-user `UserValves` overrides | +| **P4** | Conversation-level | **Chat UI tool selection** — only tools the user explicitly checks in the chat input are activated | + +**How P4 works**: OpenWebUI passes the user's currently selected tools as `tool_ids` in `__metadata__`. The plugin reads this whitelist and filters both **OpenWebUI tools** and **MCP servers** accordingly. + +- **Default ON (No Selection)**: If you **do not select any tools** (leave the checkbox empty), the plugin considers all enabled tools active (provided the P1 global Valve is enabled). OpenWebUI tools, OpenAPI Servers, and MCP Servers are **all mounted by default**. +- **Whitelist Mode (Selection Active)**: Once you explicitly check at least one tool (e.g., only "web search"), **only your selected tools are activated**, and all other tools (including MCP servers) are filtered out. + +``` +User checks ✅ web-search, ✅ code-runner in Chat UI + ↓ +chat_tool_ids = ["web-search", "server:code-runner"] + ↓ +filtered tools = only the two checked tools + ↓ +MCP servers not in the list are skipped entirely +``` + +**P2 admin control**: Each tool server entry in OpenWebUI now has a `config.enable` field. Setting it to `false` from the admin panel disables that server globally — no Valve edits required. + +### 2. File Publish Reliability — All Storage Backends Fixed + +The persistent `{"detail":"[ERROR: Error getting file content]"}` error when agents publish files has been fully resolved. + +| Root Cause | Fix | +| :--- | :--- | +| Fallback path used `shutil.copy2` + manual DB write, which stored a local absolute path that S3-backed deployments cannot resolve | Fallback now calls `Storage.upload_file()` directly, auto-adapting to local/S3/GCS/Azure | +| HTML files were blocked by OpenWebUI's `ALLOWED_FILE_EXTENSIONS` check | Upload URL always includes `?process=false`, bypassing the content-type filter | + +When the published artifact is an HTML file, the plugin also returns a direct-access HTML link for immediate opening/preview in chat. + +All published links now follow a strict format: they must be relative paths beginning with `/api/v1/files/`. +`api/...` (without leading slash) and domain-prefixed absolute URLs are treated as invalid. + +### 3. CLI Built-in Tools Always Available + +`available_tools` is now always `None` (instead of filtering by loaded tool IDs), which means Copilot CLI built-in tools — such as `bash`, `create_file`, `read_file`, `list_directory` — are **always available by default**. + +Previously, when no server tools were configured/loaded, the CLI could receive `available_tools=[]` and silently block all built-ins. + +### 4. Publish Tool Always Injected + +`publish_file_from_workspace` is injected into the tool list before the `ENABLE_OPENWEBUI_TOOLS` guard, so it is **never lost** even when all OpenWebUI tool loading is disabled via Valves. + +### 5. Code Interpreter Warning (Not Disabled) + +The built-in `code_interpreter` tool is **enabled** but operates in a remote, ephemeral sandbox. + +A system prompt warning is now automatically injected to clarify its limitations: +> "The `code_interpreter` tool executes code in a remote, ephemeral environment. It cannot access files in your local workspace or persist changes." + +### 6. Bug Fixes Summary + +- **File publish backend mismatch**: Fixed object-storage publish failures (`Error getting file content`) caused by local-path fallback writes. +- **HTML artifact extension rejection**: Fixed HTML uploads being blocked by OpenWebUI extension processing by forcing `?process=false`. +- **Invalid artifact URL formats**: Fixed links occasionally generated as `api/...` or domain-prefixed URLs; output is now constrained to `/api/v1/files/...` relative paths. +- **CLI built-ins silently blocked**: Fixed built-ins becoming unavailable when no server tools were configured/loaded (which resulted in `available_tools=[]`); now default remains `None`. +- **Publish tool injection loss**: Fixed `publish_file_from_workspace` being omitted when `ENABLE_OPENWEBUI_TOOLS=False`. + +### 7. Companion Files Filter Included in Pipe Guidance + +The release documentation now embeds the key points of `GitHub Copilot SDK Files Filter` so users can configure the full workflow from a single page: + +- It prevents default RAG pre-processing from consuming uploaded files before Pipe processing. +- It preserves raw binary access by moving uploads into `copilot_files`. +- v0.1.3 improvements included in guidance: BYOK model-id prefix matching + optional dual-channel debug logs. + +--- + +## 🛠️ Key Capabilities + +| Feature | Description | +| :--- | :--- | +| **Conditional Tool Filtering (P1~P4)** | Multi-priority permission system: global Valves → admin config.enable → user valves → Chat UI selection | +| **Universal Tool Protocol** | Native support for **MCP**, **OpenAPI**, and **OpenWebUI built-in tools** | +| **Native Tool Call UI** | Adapted to OpenWebUI's built-in tool call rendering | +| **Workspace Isolation** | Strict sandboxing for per-session data privacy and security | +| **Workspace Artifacts** | Agents generate files (Excel/CSV/HTML) with persistent download links | +| **Multi-storage Publishing** | File publish works across local disk, S3, GCS, and Azure backends | +| **11-Language Localization** | Auto-detected, native status messages for global users | + +--- + +## 🔄 Migration Notes + +- **No breaking changes** for existing users. The P4 whitelist is only active when the user has explicitly selected tools in the Chat UI; with no selection, all enabled tools are passed as before. +- Users on S3/GCS/Azure deployments who experienced file download failures can now publish normally without any additional configuration. +- `ENABLE_TOOL_CACHE` Valve has been removed. Tool server connections are now always read fresh from the database to avoid stale state across multiple workers. Use `MODEL_CACHE_TTL` for model list caching. + +--- + +## 📥 Import Chat Templates + +- [📥 Star Prediction Chat log](https://fu-jie.github.io/awesome-openwebui/plugins/pipes/star-prediction-chat.json) +- [📥 Video Processing Chat log](https://fu-jie.github.io/awesome-openwebui/plugins/pipes/video-processing-chat.json) + +*Settings → Data → Import Chats.* + +--- + +## 🔗 Resources + +- **GitHub Repository**: [openwebui-extensions](https://github.com/Fu-Jie/openwebui-extensions) +- **Full Changelog**: [README.md](https://github.com/Fu-Jie/openwebui-extensions/blob/main/plugins/pipes/github-copilot-sdk/README.md) diff --git a/plugins/pipes/github-copilot-sdk/v0.8.0_CN.md b/plugins/pipes/github-copilot-sdk/v0.8.0_CN.md new file mode 100644 index 0000000..cb944f0 --- /dev/null +++ b/plugins/pipes/github-copilot-sdk/v0.8.0_CN.md @@ -0,0 +1,131 @@ +# 🚀 GitHub Copilot SDK Pipe v0.8.0：条件工具过滤 & 发布可靠性全面升级 🎛️ + +**GitHub Copilot SDK Pipe v0.8.0** — 一次重大的权限管控与稳定性升级。本次更新引入了四优先级工具权限体系，使工具访问可按每次对话精细配置；同时彻底修复了所有存储后端的文件发布问题，并确保 CLI 内置工具不再被意外屏蔽。 + +--- + +## 📦 快速安装 + +- **GitHub Copilot SDK (Pipe 插件)**: [安装 v0.8.0](https://openwebui.com/posts/ce96f7b4-12fc-4ac3-9a01-875713e69359) +- **GitHub Copilot SDK (Filter 插件)**: [安装 v0.1.3](https://openwebui.com/posts/403a62ee-a596-45e7-be65-fab9cc249dd6) + +--- + +## 🚀 v0.8.0 更新内容 + +### 1. 条件工具过滤 (P1~P4) — 核心新特性 + +本次更新引入了**四优先级工具权限体系**，为您提供对每次对话中可用工具的精确控制。 + +| 优先级 | 作用域 | 机制描述 | +| :--- | :--- | :--- | +| **P1** | 全局 | Pipe 级 Valve 开关（`ENABLE_OPENWEBUI_TOOLS`、`ENABLE_OPENAPI_SERVER`、`ENABLE_MCP`） | +| **P2** | 服务器级 | 管理员 `config.enable` — 无需修改代码，直接从 OpenWebUI 管理面板禁用某个工具服务器 | +| **P3** | 用户级 | 用户专属 `UserValves` 覆盖设置 | +| **P4** | 对话级 | **Chat UI 工具选择** — 只有用户在输入框明确勾选的工具才会在本次对话中激活 | + +**P4 工作原理**：OpenWebUI 会将用户当前选中的工具 ID 通过 `__metadata__` 中的 `tool_ids` 字段传入。插件读取该白名单并对 **OpenWebUI 工具**和 **MCP server** 同时进行过滤。 + +- **默认全开 (Default ON)**：如果您在 Chat UI 中**未勾选任何工具**（留空），只要 P1 全局 Valve 开启，插件就会默认启用**所有可用工具**。OpenWebUI 工具、OpenAPI Server 和 MCP Server 将默认全部挂载。 +- **按需激活 (Whitelist Mode)**：一旦您在输入框主动勾选了至少一个工具（例如只选了“联网搜索”），则**只有您选中的工具会生效**，其他未选中的工具（包括 MCP 服务器）都将被过滤掉。 + +``` +用户在 Chat UI 勾选 ✅ web-search，✅ code-runner + ↓ +chat_tool_ids = ["web-search", "server:code-runner"] + ↓ +过滤后工具列表 = 仅包含上述两个工具 + ↓ +白名单外的 MCP server 将被完全跳过 +``` + +**P2 管理员管控**：OpenWebUI 中每个工具服务器条目现在都有 `config.enable` 字段。从管理面板将其设为 `false` 即可全局禁用该服务器，无需修改任何 Valve 配置。 + +### 2. 文件发布全面修复 — 适配所有存储后端 + +Agent 发布文件时持续出现的 `{"detail":"[ERROR: Error getting file content]"}` 错误已被彻底解决。 + +| 根本原因 | 修复方案 | +| :--- | :--- | +| 回退路径使用 `shutil.copy2` + 手动写入 DB，导致 `file.path` 存储的是本地绝对路径，S3 等对象存储后端无法解析 | 回退路径改为直接调用 `Storage.upload_file()`，自动适配 local/S3/GCS/Azure | +| HTML 文件被 OpenWebUI `ALLOWED_FILE_EXTENSIONS` 内容类型检查拦截 | 上传 URL 始终追加 `?process=false`，绕过文件类型限制 | + +当发布产物为 HTML 文件时，插件还会返回一个可直接访问的 HTML 链接，便于在聊天中即时打开/预览。 + +所有发布链接现统一执行严格格式：必须是以 `/api/v1/files/` 开头的相对路径。 +`api/...`（缺少前导 `/`）或拼接域名的绝对 URL 都视为无效格式。 + +### 3. CLI 内置工具始终可用 + +`available_tools` 现在统一设为 `None`（不再根据已加载的工具 ID 列表过滤），这意味着 Copilot CLI 内置工具 —— 如 `bash`、`create_file`、`read_file`、`list_directory` —— **默认始终可用**。 + +此前，在未配置/未加载任何 server 工具时，CLI 可能收到 `available_tools=[]`，导致所有内置工具被静默屏蔽。 + +### 4. 发布工具始终注入 + +`publish_file_from_workspace` 工具的注入逻辑提前到 `ENABLE_OPENWEBUI_TOOLS` 守卫之前，因此即使通过 Valve 关闭所有 OpenWebUI 工具加载，该工具也**不再丢失**。 + +### 5. 代码解释器警告（未禁用） + +内置的 `code_interpreter` 工具保持**启用**状态，但运行在远程的临时沙箱中。 + +为了避免混淆，系统提示词中会自动注入一段警告说明： +> "The `code_interpreter` tool executes code in a remote, ephemeral environment. It cannot access files in your local workspace or persist changes." + +请仅将其用于纯计算或逻辑验证，勿用于文件操作。 + +### 6. Bug 修复汇总 + +- **文件发布后端不匹配**：修复了对象存储场景下因本地路径回退写入导致的 `Error getting file content`。 +- **HTML 产物扩展名拦截**：修复了 OpenWebUI 扩展名处理导致 HTML 上传被拒的问题，统一强制 `?process=false`。 +- **产物链接格式错误**：修复了链接偶发被生成成 `api/...` 或带域名 URL 的问题，现统一限制为 `/api/v1/files/...` 相对路径。 +- **CLI 内置工具被静默屏蔽**：修复了在未配置/未加载任何 server 工具时（最终出现 `available_tools=[]`）导致内置工具不可用的问题，默认保持 `None`。 +- **发布工具注入丢失**：修复了 `ENABLE_OPENWEBUI_TOOLS=False` 时 `publish_file_from_workspace` 被遗漏的问题。 + +### 7. 在 Pipe 文档中整合 Files Filter 指南 + +本次已将 `GitHub Copilot SDK Files Filter` 的核心说明整合进 Pipe 文档，用户可在单页完成完整配置： + +- 阻止默认 RAG 在 Pipe 接手前抢先处理上传文件。 +- 通过将上传文件移动到 `copilot_files` 保留原始二进制访问能力。 +- 纳入 v0.1.3 重点：BYOK 模型 ID 前缀匹配修复 + 可选双通道调试日志。 + +--- + +## 🛠️ 核心能力 + +| 特性 | 描述 | +| :--- | :--- | +| **条件工具过滤 (P1~P4)** | 多优先级权限体系：全局 Valve → 管理员 config.enable → 用户 Valve → Chat UI 选择 | +| **通用工具协议** | 原生支持 **MCP**、**OpenAPI** 以及 **OpenWebUI 内置工具** | +| **原生工具 UI** | 完美适配 OpenWebUI 内置的工具调用渲染 | +| **物理隔离工作区** | 为每个会话提供严格的沙箱环境，保护数据隐私与安全 | +| **工作区产物工具** | Agent 可生成文件（Excel/CSV/HTML）并提供持久化下载链接 | +| **多存储后端发布** | 文件发布兼容本地磁盘、S3、GCS 及 Azure 后端 | +| **11 国语言本地化** | 自动检测并显示原生语言的状态消息 | + +--- + +## 🔄 迁移说明 + +- **无破坏性变更**。P4 白名单过滤仅在用户主动在 Chat UI 选择了工具时生效；未做任何选择时，所有已启用的工具照常传递给 Copilot。 +- 使用 S3/GCS/Azure 存储后端、曾遭遇文件下载失败的用户，无需任何额外配置即可正常发布文件。 +- `ENABLE_TOOL_CACHE` Valve 已移除。工具服务器连接现在始终从数据库实时读取，避免多 Worker 环境下的状态过时。模型列表缓存请使用 `MODEL_CACHE_TTL`。 + +--- + +## 📥 导入对话模板 + +您可以下载并导入真实案例的 JSON 对话日志，直观查看模型如何调用工具： + +- [📥 对话日志：GitHub 增长预测](https://fu-jie.github.io/awesome-openwebui/plugins/pipes/star-prediction-chat.json) +- [📥 对话日志：视频高质量 GIF 转换](https://fu-jie.github.io/awesome-openwebui/plugins/pipes/video-processing-chat.json) + +*导入方式：前往 `设置 → 数据 → 导入对话`。* + +--- + +## 🔗 相关资源 + +- **GitHub 仓库**: [openwebui-extensions](https://github.com/Fu-Jie/openwebui-extensions) +- **完整变更日志**: [README_CN.md](https://github.com/Fu-Jie/openwebui-extensions/blob/main/plugins/pipes/github-copilot-sdk/README_CN.md)