Compare commits
10 Commits
async-cont
...
async-cont
| Author | SHA1 | Date | |
|---|---|---|---|
|
cd95b5ff69 | ||
|
|
3210262296 | ||
|
|
37a130993a | ||
|
|
b75fd96e4a | ||
|
|
5dd9d6cc56 | ||
|
|
d569dc3ec9 | ||
|
|
e2426c74e1 | ||
|
|
ae0fa1d39a | ||
|
|
62e78ace5c | ||
|
|
7efb64b16b |
@@ -0,0 +1,27 @@
|
||||
# Async Context Compression Progress Mapping
|
||||
|
||||
> Discovered: 2026-03-10
|
||||
|
||||
## Context
|
||||
Applies to `plugins/filters/async-context-compression/async_context_compression.py` once the inlet has already replaced early history with a synthetic summary message.
|
||||
|
||||
## Finding
|
||||
`compressed_message_count` cannot be recalculated from the visible message list length after compression. Once a summary marker is present, the visible list mixes:
|
||||
- preserved head messages that are still before the saved boundary
|
||||
- one synthetic summary message
|
||||
- tail messages that map to original history starting at the saved boundary
|
||||
|
||||
## Solution / Pattern
|
||||
Store the original-history boundary on the injected summary message metadata, then recover future progress using:
|
||||
- `original_count = covered_until + len(messages_after_summary_marker)`
|
||||
- `target_progress = max(covered_until, original_count - keep_last)`
|
||||
|
||||
When the summary-model window is too small, trim newest atomic groups from the summary input so the saved boundary still matches what the summary actually covers.
|
||||
|
||||
## Gotchas
|
||||
- If you trim from the head of the summary input, the saved progress can overstate coverage and hide messages that were never summarized.
|
||||
- Status previews for the next context must convert the saved original-history boundary back into the current visible view before rebuilding head/summary/tail.
|
||||
- `inlet(body["messages"])` and `outlet(body["messages"])` can both represent the full conversation while using different serializations:
|
||||
- inlet may receive expanded native tool-call chains (`assistant(tool_calls) -> tool -> assistant`)
|
||||
- outlet may receive a compact top-level transcript where tool calls are folded into assistant `<details type="tool_calls">` blocks
|
||||
- These two views do not share a safe `compressed_message_count` coordinate system. If outlet is in the compact assistant/details view, do not persist summary progress derived from its top-level message count.
|
||||
26
.agent/learnings/openwebui-tool-call-context-inflation.md
Normal file
26
.agent/learnings/openwebui-tool-call-context-inflation.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# OpenWebUI Tool Call Context Inflation
|
||||
|
||||
> Discovered: 2026-03-11
|
||||
|
||||
## Context
|
||||
When analyzing why the `async_context_compression` plugin sees different array lengths of `messages` between the `inlet` (e.g. 27 items) and `outlet` (e.g. 8 items) phases, especially when native tool calling (Function Calling) is involved in OpenWebUI.
|
||||
|
||||
## Finding
|
||||
There is a fundamental disparity in how OpenWebUI serializes conversational history at different stages of the request lifecycle:
|
||||
|
||||
1. **Outlet (UI Rendering View)**:
|
||||
After the LLM completes generation and tools have been executed, OpenWebUI's `middleware.py` (and streaming builders) bundles intermediate tool calls and their raw results. It hides them inside an HTML `<details type="tool_calls">...</details>` block within a single `role: assistant` message's `content`.
|
||||
Concurrently, the actual native API tool-calling data is saved in a hidden `output` dict field attached to that message. At this stage, the `messages` array looks short (e.g., 8 items) because tool interactions are visually folded.
|
||||
|
||||
2. **Inlet (LLM Native View)**:
|
||||
When the user sends the *next* message, the request enters `main.py` -> `process_chat_payload` -> `middleware.py:process_messages_with_output()`.
|
||||
Here, OpenWebUI scans historical `assistant` messages for that hidden `output` field. If found, it completely **inflates (unfolds)** the raw data back into an exact sequence of OpenAI-compliant `tool_call` and `tool_result` messages (using `utils/misc.py:convert_output_to_messages`).
|
||||
The HTML `<details>` string is entirely discarded before being sent to the LLM.
|
||||
|
||||
**Conclusion on Token Consumption**:
|
||||
In the next turn, tool context is **NOT** compressed at all. It is fully re-expanded to its original verbose state (e.g., back to 27 items) and consumes the maximum amount of tokens required by the raw JSON arguments and results.
|
||||
|
||||
## Gotchas
|
||||
- Any logic operating in the `outlet` phase (like background tasks) that relies on the `messages` array index will be completely misaligned with the array seen in the `inlet` phase.
|
||||
- Attempting to slice or trim history based on `outlet` array lengths will cause index out-of-bounds errors or destructive cropping of recent messages.
|
||||
- The only safe way to bridge these two views is either to translate the folded view back into the expanded view using `convert_output_to_messages`, or to rely on unique `id` fields (if available) rather than array indices.
|
||||
@@ -73,13 +73,21 @@ Create two versioned release notes files:
|
||||
#### Required Sections
|
||||
|
||||
Each file must include:
|
||||
0. **Marketplace Link**: Direct link to the plugin on openwebui.com (e.g., `**[🚀 Get/Update on OpenWebUI Community](URL)**`)
|
||||
1. **Overview**: One paragraph summarizing this release
|
||||
2. **New Features** / **新功能**: Bulleted list of features
|
||||
3. **Bug Fixes** / **问题修复**: Bulleted list of fixes
|
||||
4. **Related Issues** / **相关 Issue**: Link to the GitHub Issue(s) resolved in this release (e.g., `**[#56](URL)**`). MANDATORY if the release resolves a reported issue.
|
||||
5. **Related PRs** / **相关 PR**: Link to the Pull Request(s) associated with this release. (e.g., `**[#123](URL)**`). MANDATORY if the release is being prepared within an existing PR.
|
||||
6. **Migration Notes** / **迁移说明**: Breaking changes or Valve key renames (omit section if none)
|
||||
0. **Marketplace Badge**: A prominent button linking to the plugin on openwebui.com using shields.io (e.g., `[](URL)`).
|
||||
1. **Overview Header**: Use `## Overview` as the first header.
|
||||
2. **Summary Paragraph**: A paragraph summarizing the release. **NEVER** include the version number as a title.
|
||||
3. **README Link**: Direct link to the plugin's README file on GitHub.
|
||||
4. **New Features** / **新功能**: Bulleted list of features
|
||||
5. **Bug Fixes** / **问题修复**: Bulleted list of fixes
|
||||
6. **Related Issues** / **相关 Issue**: Link to GitHub Issues. **ONLY** include if a specific issue is resolved. **NEVER use placeholders.**
|
||||
7. **Related PRs** / **相关 PR**: Link to the Pull Request. **ONLY** include if the PR is already created and the ID is known. **NEVER use placeholders.**
|
||||
8. **Migration Notes**: Breaking changes or Valve key renames (omit section if none)
|
||||
|
||||
---
|
||||
|
||||
## Language Standard
|
||||
|
||||
- **Release Notes Files**: Use **English ONLY** for the final `.md` files to maintain professional consistency on GitHub. Avoid bilingual content in the release description.
|
||||
6. **Companion Plugins** / **配套插件** (optional): If a companion plugin was updated
|
||||
|
||||
If a release notes file already exists for this version, update it rather than creating a new one.
|
||||
|
||||
23
.github/agents/release-prep.agent.md
vendored
23
.github/agents/release-prep.agent.md
vendored
@@ -78,5 +78,28 @@ Plugin: {type}/{name} → v{new_version}
|
||||
### Verification Status
|
||||
{filled-in 9-file checklist for each changed plugin}
|
||||
|
||||
## Post-Release: Batch Plugin Installation
|
||||
|
||||
After release is published, users can quickly install all plugins:
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/Fu-Jie/openwebui-extensions.git
|
||||
cd openwebui-extensions
|
||||
|
||||
# Setup API key and instance URL
|
||||
echo "api_key=sk-your-api-key-here" > scripts/.env
|
||||
echo "url=http://localhost:3000" >> scripts/.env
|
||||
|
||||
# If using remote instance, configure the baseURL:
|
||||
# echo "url=http://192.168.1.10:3000" >> scripts/.env
|
||||
# echo "url=https://openwebui.example.com" >> scripts/.env
|
||||
|
||||
# Install all plugins at once
|
||||
python scripts/install_all_plugins.py
|
||||
```
|
||||
|
||||
See: [Deployment Guide](./scripts/DEPLOYMENT_GUIDE.md)
|
||||
|
||||
---
|
||||
⚠️ **Waiting for user confirmation — no git operations will run until explicitly approved.**
|
||||
|
||||
55
.github/workflows/release.yml
vendored
55
.github/workflows/release.yml
vendored
@@ -436,54 +436,34 @@ jobs:
|
||||
CHANGED_PLUGIN_TITLE: ${{ needs.check-changes.outputs.changed_plugin_title }}
|
||||
CHANGED_PLUGIN_VERSION: ${{ needs.check-changes.outputs.changed_plugin_version }}
|
||||
DETECTED_CHANGES: ${{ needs.check-changes.outputs.release_notes }}
|
||||
COMMITS: ${{ steps.commits.outputs.commits }}
|
||||
DOC_FILES: ${{ needs.check-changes.outputs.changed_doc_files }}
|
||||
run: |
|
||||
> release_notes.md
|
||||
|
||||
if [ -n "$CHANGED_PLUGIN_TITLE" ] && [ -n "$CHANGED_PLUGIN_VERSION" ]; then
|
||||
echo "# $CHANGED_PLUGIN_TITLE v$CHANGED_PLUGIN_VERSION" >> release_notes.md
|
||||
echo "" >> release_notes.md
|
||||
elif [ -n "$TITLE" ]; then
|
||||
echo "# $TITLE" >> release_notes.md
|
||||
echo "" >> release_notes.md
|
||||
fi
|
||||
|
||||
# 1. Release notes from v*.md files (highest priority, shown first)
|
||||
# 1. Primary content from v*.md files (highest priority)
|
||||
if [ -n "$DOC_FILES" ]; then
|
||||
RELEASE_NOTE_FILES=$(echo "$DOC_FILES" | grep -E '^plugins/.*/v[^/]*\.md$' | grep -v '_CN\.md$' || true)
|
||||
if [ -n "$RELEASE_NOTE_FILES" ]; then
|
||||
while IFS= read -r file; do
|
||||
[ -z "$file" ] && continue
|
||||
if [ -f "$file" ]; then
|
||||
python3 -c "import pathlib, re; file_path = pathlib.Path(r'''$file'''); text = file_path.read_text(encoding='utf-8'); text = re.sub(r'^#\\s+.+?(?:\\r?\\n)+', '', text, count=1, flags=re.MULTILINE); print(text.lstrip().rstrip())" >> release_notes.md
|
||||
# Extract content, removing any H1 title from the file to avoid duplication
|
||||
python3 -c "import pathlib, re; file_path = pathlib.Path(r'''$file'''); text = file_path.read_text(encoding='utf-8'); text = re.sub(r'^#\s+.+?(?:\r?\n)+', '', text, count=1, flags=re.MULTILINE); print(text.lstrip().rstrip())" >> release_notes.md
|
||||
echo "" >> release_notes.md
|
||||
fi
|
||||
done <<< "$RELEASE_NOTE_FILES"
|
||||
fi
|
||||
fi
|
||||
|
||||
# 2. Plugin version changes detected by script
|
||||
if [ -z "$CHANGED_PLUGIN_TITLE" ] && [ -z "$CHANGED_PLUGIN_VERSION" ] && [ -n "$TITLE" ]; then
|
||||
echo "## $TITLE" >> release_notes.md
|
||||
echo "" >> release_notes.md
|
||||
fi
|
||||
|
||||
# 2. Automated plugin version change summary
|
||||
if [ -n "$DETECTED_CHANGES" ] && ! echo "$DETECTED_CHANGES" | grep -q "No changes detected"; then
|
||||
echo "## What's Changed" >> release_notes.md
|
||||
echo "## Version Changes" >> release_notes.md
|
||||
echo "" >> release_notes.md
|
||||
echo "$DETECTED_CHANGES" >> release_notes.md
|
||||
echo "" >> release_notes.md
|
||||
fi
|
||||
|
||||
# 3. Commits (Conventional Commits format with body)
|
||||
if [ -n "$COMMITS" ]; then
|
||||
echo "## Commits" >> release_notes.md
|
||||
echo "" >> release_notes.md
|
||||
echo "$COMMITS" >> release_notes.md
|
||||
echo "" >> release_notes.md
|
||||
fi
|
||||
|
||||
# 3. Manual additional notes from workflow dispatch
|
||||
if [ -n "$NOTES" ]; then
|
||||
echo "## Additional Notes" >> release_notes.md
|
||||
echo "" >> release_notes.md
|
||||
@@ -493,30 +473,15 @@ jobs:
|
||||
|
||||
cat >> release_notes.md << 'EOF'
|
||||
|
||||
## Download
|
||||
|
||||
📦 **Download the updated plugin files below**
|
||||
|
||||
### Installation
|
||||
|
||||
#### From OpenWebUI Community
|
||||
1. Open OpenWebUI Admin Panel
|
||||
2. Navigate to Functions/Tools
|
||||
3. Search for the plugin name
|
||||
4. Click Install
|
||||
|
||||
#### Manual Installation
|
||||
1. Download the plugin file (`.py`) from the assets below
|
||||
2. Open OpenWebUI Admin Panel → Functions
|
||||
3. Click "Create Function" → Import
|
||||
4. Paste the plugin code
|
||||
|
||||
---
|
||||
|
||||
📚 [Documentation](https://fu-jie.github.io/openwebui-extensions/)
|
||||
📚 [Documentation Portal](https://fu-jie.github.io/openwebui-extensions/)
|
||||
🐛 [Report Issues](https://github.com/Fu-Jie/openwebui-extensions/issues)
|
||||
EOF
|
||||
|
||||
echo "=== Final Release Notes ==="
|
||||
cat release_notes.md
|
||||
|
||||
echo "=== Release Notes ==="
|
||||
cat release_notes.md
|
||||
|
||||
|
||||
75
CHANGELOG.md
75
CHANGELOG.md
@@ -1,75 +0,0 @@
|
||||
# Changelog / 更新日志
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
本项目的所有重要更改都将记录在此文件中。
|
||||
格式基于 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)，
|
||||
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
|
||||
|
||||
---
|
||||
|
||||
## [Unreleased] / 未发布
|
||||
|
||||
### Added / 新增
|
||||
- 插件发布工作流 (Plugin release workflow)
|
||||
|
||||
### Changed / 变更
|
||||
|
||||
### Fixed / 修复
|
||||
|
||||
### Removed / 移除
|
||||
|
||||
---
|
||||
|
||||
## Plugin Versions / 插件版本
|
||||
|
||||
### Actions
|
||||
|
||||
| Plugin / 插件 | Version / 版本 |
|
||||
|---------------|----------------|
|
||||
| Smart Mind Map / 思维导图 | 0.8.0 |
|
||||
| Flash Card / 闪记卡 | 0.2.1 |
|
||||
| Export to Word / 导出为 Word | 0.1.0 |
|
||||
| Export to Excel / 导出为 Excel | 0.3.3 |
|
||||
| Deep Reading & Summary / 精读 | 0.1.0 / 2.0.0 |
|
||||
| Smart Infographic / 智能信息图 | 1.3.0 |
|
||||
|
||||
### Filters
|
||||
|
||||
| Plugin / 插件 | Version / 版本 |
|
||||
|---------------|----------------|
|
||||
| Async Context Compression / 异步上下文压缩 | 1.1.0 |
|
||||
| Context & Model Enhancement Filter | 0.2 |
|
||||
| Gemini Manifold Companion | 1.7.0 |
|
||||
| Gemini 多模态过滤器 | 0.3.2 |
|
||||
|
||||
### Pipes
|
||||
|
||||
| Plugin / 插件 | Version / 版本 |
|
||||
|---------------|----------------|
|
||||
| Gemini Manifold google_genai | 1.26.0 |
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
Release Template / 发布模板:
|
||||
|
||||
## [x.x.x] - YYYY-MM-DD
|
||||
|
||||
### Added / 新增
|
||||
- New feature description
|
||||
|
||||
### Changed / 变更
|
||||
- Change description
|
||||
|
||||
### Fixed / 修复
|
||||
- Bug fix description
|
||||
|
||||
### Plugin Updates / 插件更新
|
||||
- `plugin_name`: v0.x.0 -> v0.y.0
|
||||
- Feature 1
|
||||
- Feature 2
|
||||
-->
|
||||
@@ -1,104 +0,0 @@
|
||||
# Markdown Normalizer 插件可靠性修复分析报告 (Issue #57)
|
||||
|
||||
## 1. 问题背景
|
||||
根据 Issue #57 报告，`Markdown Normalizer` 在 v1.2.7 版本中存在数项严重影响可靠性的 Bug，包括错误回滚失效、对内联技术内容的过度转义、配置项不生效以及调试日志潜在的隐私风险。
|
||||
|
||||
## 2. 核心处理流程图 (v1.2.8)
|
||||
以下流程展示了插件如何在确保“不损坏原始内容”的前提下进行智能修复：
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
Start([开始处理内容]) --> Cache[1. 内存中存入原始快照 Snapshot]
|
||||
Cache --> Logic{进入修复流程}
|
||||
|
||||
subgraph "分层保护逻辑 (Context-Aware)"
|
||||
Logic --> Block[识别并锁定 ``` 代码块]
|
||||
Block --> Inline[识别并锁定 ` 行内代码]
|
||||
Inline --> Math[识别并锁定 $ LaTeX 公式]
|
||||
Math --> Clean[仅对非锁定区域执行转义清理]
|
||||
end
|
||||
|
||||
Clean --> Others[执行其他规则: Thought/Details/Table等]
|
||||
Others --> Check{运行是否报错?}
|
||||
|
||||
Check -- 否 (成功) --> Success[返回修复后的内容]
|
||||
Check -- 是 (失败) --> Rollback[触发回滚: 丢弃所有修改]
|
||||
|
||||
Rollback --> Original[返回步骤1存储的原始快照]
|
||||
|
||||
Success --> End([输出结果])
|
||||
Original --> End
|
||||
```
|
||||
|
||||
## 3. 修复项详细说明
|
||||
|
||||
### 2.1 错误回滚机制修复 (Reliability: Error Fallback)
|
||||
- **问题**：在 `normalize` 流程中，如果某个清理器抛出异常，返回的是已被部分修改的 `content`，导致输出内容损坏。
|
||||
- **技术实现**：
|
||||
```python
|
||||
def normalize(self, content: str) -> str:
|
||||
original_content = content # 1. 流程开始前缓存原始快照
|
||||
try:
|
||||
# ... 执行一系列清理步骤 ...
|
||||
return content
|
||||
except Exception as e:
|
||||
# 2. 任何步骤失败，立即记录日志并回滚
|
||||
logger.error(f"Content normalization failed: {e}", exc_info=True)
|
||||
return original_content # 确保返回的是原始快照
|
||||
```
|
||||
- **验证结果**：通过模拟 `RuntimeError` 验证，插件现在能 100% 回滚至原始状态。
|
||||
|
||||
### 2.2 上下文感知的转义保护 (Context-Aware Escaping)
|
||||
- **问题**：全局替换导致正文中包含在 `` ` `` 内的代码片段（如正则、Windows 路径）被破坏。
|
||||
- **技术实现**：
|
||||
重构后的 `_fix_escape_characters` 采用了 **“分词保护策略”**，通过多层嵌套分割来确保仅在非代码上下文中进行清理：
|
||||
```python
|
||||
def _fix_escape_characters(self, content: str) -> str:
|
||||
# 层级 1: 以 ``` 分隔代码块
|
||||
parts = content.split("```")
|
||||
for i in range(len(parts)):
|
||||
is_code_block = (i % 2 != 0)
|
||||
if is_code_block and not self.config.enable_escape_fix_in_code_blocks:
|
||||
continue # 默认跳过代码块
|
||||
|
||||
if not is_code_block:
|
||||
# 层级 2: 在非代码块正文中，以 ` 分隔内联代码
|
||||
inline_parts = parts[i].split("`")
|
||||
for k in range(0, len(inline_parts), 2): # 仅处理非内联代码部分
|
||||
# 层级 3: 在非内联代码中，以 $ 分隔 LaTeX 公式
|
||||
sub_parts = inline_parts[k].split("$")
|
||||
for j in range(0, len(sub_parts), 2):
|
||||
# 最终：仅在确认为“纯文本”的部分执行 clean_text
|
||||
sub_parts[j] = clean_text(sub_parts[j])
|
||||
inline_parts[k] = "$".join(sub_parts)
|
||||
parts[i] = "`".join(inline_parts)
|
||||
else:
|
||||
parts[i] = clean_text(parts[i])
|
||||
return "```".join(parts)
|
||||
```
|
||||
- **验证结果**：测试用例 `Regex: [\n\r]` 和 `C:\Windows` 在正文中保持原样，而普通文本中的 `\\n` 被正确转换。
|
||||
|
||||
### 2.3 配置项激活 (Configuration Enforcement)
|
||||
- **问题**：`enable_escape_fix_in_code_blocks` 开关在代码中被定义但未被逻辑引用。
|
||||
- **修复方案**：在 `_fix_escape_characters` 处理流程中加入对该开关的判断。
|
||||
- **验证结果**：当开关关闭（默认）时，代码块内容保持不变；开启时，代码块内执行转义修复。
|
||||
|
||||
### 2.4 默认日志策略调整 (Privacy & Performance)
|
||||
- **问题**：`show_debug_log` 默认为 `True`，且会将原始内容打印到浏览器控制台。
|
||||
- **修复方案**：将默认值改为 `False`。
|
||||
- **验证结果**：新安装或默认配置下不再主动输出全量日志，仅在用户显式开启时用于调试。
|
||||
|
||||
## 3. 综合测试覆盖
|
||||
已建立 `comprehensive_test_markdown_normalizer.py` 测试脚本，覆盖以下场景：
|
||||
1. **异常抛出回滚**：确保插件“不破坏”原始内容。
|
||||
2. **内联代码保护**：验证正则和路径字符串的完整性。
|
||||
3. **代码块开关控制**：验证配置项的有效性。
|
||||
4. **LaTeX 命令回归测试**：确保 `\times`, `\theta` 等命令不被误触。
|
||||
5. **复杂嵌套结构**：验证包含 Thought 标签、列表、内联代码及代码块的混合文本处理。
|
||||
|
||||
## 4. 结论
|
||||
`Markdown Normalizer v1.2.8` 已解决 Issue #57 提出的所有核心可靠性问题。插件现在具备“不损坏内容”的防御性编程能力，并能更智能地感知 Markdown 上下文。
|
||||
|
||||
---
|
||||
**报告日期**：2026-03-08
|
||||
**修复版本**：v1.2.8
|
||||
@@ -1,12 +0,0 @@
|
||||
# Reply to Issue #57
|
||||
|
||||
I have addressed these issues in **v1.2.8** with a focus on reliability and a "Safe-by-Default" approach:
|
||||
|
||||
1. **Robust Error Rollback (Items 1, 4, 5)**: I implemented a full `try...except` wrapper. If any error occurs during normalization, the plugin now returns the **100% original text**. This ensures that the output is never partially modified or corrupted.
|
||||
2. **Conservative Escaping (Item 2)**: To avoid breaking technical content like regex or paths, the escape fixer now strictly skips all code blocks, inline code, and LaTeX formulas by default. I have shifted toward an "opt-in" model where aggressive cleaning is disabled unless specifically requested.
|
||||
3. **Fixed Configuration (Item 3)**: The `enable_escape_fix_in_code_blocks` Valve was intended to handle escaping within code blocks (e.g., for fixing flat SQL output), but there was a bug preventing it from being applied. I have fixed this, and the setting is now fully functional.
|
||||
4. **Privacy & Reliability**: I have changed the default for `show_debug_log` to `False`. While it was previously enabled by default to help gather feedback and squash bugs during the initial development phase, the plugin has now undergone multiple iterations and reliability enhancements (including the new tiered protection and rollback mechanisms), making it stable enough for a "silent" and private default operation.
|
||||
|
||||
**Recommendation**: If you encounter SQL or data blocks that appear on a single line, you can now manually enable `enable_escape_fix_in_code_blocks` in the Valves to fix them safely.
|
||||
|
||||
Please update to the latest version via [OpenWebUI Community](https://openwebui.com/functions/baaa8732-9348-40b7-8359-7e009660e23c). Thank you for your valuable feedback!
|
||||
26
README.md
26
README.md
@@ -9,6 +9,7 @@ A collection of enhancements, plugins, and prompts for [open-webui](https://gith
|
||||
|
||||
<!-- STATS_START -->
|
||||
## 📊 Community Stats
|
||||
>
|
||||
> 
|
||||
|
||||
| 👤 Author | 👥 Followers | ⭐ Points | 🏆 Contributions |
|
||||
@@ -19,18 +20,19 @@ A collection of enhancements, plugins, and prompts for [open-webui](https://gith
|
||||
| :---: | :---: | :---: | :---: | :---: |
|
||||
|  |  |  |  |  |
|
||||
|
||||
|
||||
### 🔥 Top 6 Popular Plugins
|
||||
|
||||
| Rank | Plugin | Version | Downloads | Views | 📅 Updated |
|
||||
| :---: | :--- | :---: | :---: | :---: | :---: |
|
||||
| 🥇 | [Smart Mind Map](https://openwebui.com/posts/turn_any_text_into_beautiful_mind_maps_3094c59a) |  |  |  |  |
|
||||
| 🥈 | [Smart Infographic](https://openwebui.com/posts/smart_infographic_ad6f0c7f) |  |  |  |  |
|
||||
| 🥉 | [Markdown Normalizer](https://openwebui.com/posts/markdown_normalizer_baaa8732) |  |  |  |  |
|
||||
| 4️⃣ | [Export to Word Enhanced](https://openwebui.com/posts/export_to_word_enhanced_formatting_fca6a315) |  |  |  |  |
|
||||
| 5️⃣ | [Async Context Compression](https://openwebui.com/posts/async_context_compression_b1655bc8) |  |  |  |  |
|
||||
| 5️⃣ | [Async Context Compression](https://openwebui.com/posts/async_context_compression_b1655bc8) |  |  |  |  |
|
||||
| 6️⃣ | [AI Task Instruction Generator](https://openwebui.com/posts/ai_task_instruction_generator_9bab8b37) |  |  |  |  |
|
||||
|
||||
### 📈 Total Downloads Trend
|
||||
|
||||
|
||||
|
||||
*See full stats and charts in [Community Stats Report](./docs/community-stats.md)*
|
||||
@@ -163,13 +165,6 @@ For code examples, please check the `docs/examples/` directory.
|
||||
|
||||
This project is a collection of resources and does not require a Python environment. Simply download the files you need and import them into your OpenWebUI instance.
|
||||
|
||||
### Using Prompts
|
||||
|
||||
1. Browse the `/prompts` directory and select a prompt file (`.md`).
|
||||
2. Copy the file content.
|
||||
3. In the OpenWebUI chat interface, click the "Prompt" button above the input box.
|
||||
4. Paste the content and save.
|
||||
|
||||
### Using Plugins
|
||||
|
||||
1. **Install from OpenWebUI Community (Recommended)**:
|
||||
@@ -177,11 +172,14 @@ This project is a collection of resources and does not require a Python environm
|
||||
- Browse the plugins and select the one you like.
|
||||
- Click "Get" to import it directly into your OpenWebUI instance.
|
||||
|
||||
2. **Manual Installation**:
|
||||
- Browse the `/plugins` directory and download the plugin file (`.py`) you need.
|
||||
- Go to OpenWebUI **Admin Panel** -> **Settings** -> **Plugins**.
|
||||
- Click the upload button and select the `.py` file you just downloaded.
|
||||
- Once uploaded, refresh the page to enable the plugin in your chat settings or toolbar.
|
||||
2. **Quick Install All Plugins**: To install all plugins to your local OpenWebUI instance at once, clone this repo and run `python scripts/install_all_plugins.py` after configuring your API key in `.env` — see [Deployment Guide](./scripts/DEPLOYMENT_GUIDE.md) for details.
|
||||
|
||||
### Using Prompts
|
||||
|
||||
1. Browse the `/prompts` directory and select a prompt file (`.md`).
|
||||
2. Copy the file content.
|
||||
3. In the OpenWebUI chat interface, click the "Prompt" button above the input box.
|
||||
4. Paste the content and save.
|
||||
|
||||
### Contributing
|
||||
|
||||
|
||||
22
README_CN.md
22
README_CN.md
@@ -6,6 +6,7 @@ OpenWebUI 增强功能集合。包含个人开发与收集的插件、提示词
|
||||
|
||||
<!-- STATS_START -->
|
||||
## 📊 社区统计
|
||||
>
|
||||
> 
|
||||
|
||||
| 👤 作者 | 👥 粉丝 | ⭐ 积分 | 🏆 贡献 |
|
||||
@@ -16,18 +17,19 @@ OpenWebUI 增强功能集合。包含个人开发与收集的插件、提示词
|
||||
| :---: | :---: | :---: | :---: | :---: |
|
||||
|  |  |  |  |  |
|
||||
|
||||
|
||||
### 🔥 热门插件 Top 6
|
||||
|
||||
| 排名 | 插件 | 版本 | 下载 | 浏览 | 📅 更新 |
|
||||
| :---: | :--- | :---: | :---: | :---: | :---: |
|
||||
| 🥇 | [Smart Mind Map](https://openwebui.com/posts/turn_any_text_into_beautiful_mind_maps_3094c59a) |  |  |  |  |
|
||||
| 🥈 | [Smart Infographic](https://openwebui.com/posts/smart_infographic_ad6f0c7f) |  |  |  |  |
|
||||
| 🥉 | [Markdown Normalizer](https://openwebui.com/posts/markdown_normalizer_baaa8732) |  |  |  |  |
|
||||
| 4️⃣ | [Export to Word Enhanced](https://openwebui.com/posts/export_to_word_enhanced_formatting_fca6a315) |  |  |  |  |
|
||||
| 5️⃣ | [Async Context Compression](https://openwebui.com/posts/async_context_compression_b1655bc8) |  |  |  |  |
|
||||
| 5️⃣ | [Async Context Compression](https://openwebui.com/posts/async_context_compression_b1655bc8) |  |  |  |  |
|
||||
| 6️⃣ | [AI Task Instruction Generator](https://openwebui.com/posts/ai_task_instruction_generator_9bab8b37) |  |  |  |  |
|
||||
|
||||
### 📈 总下载量累计趋势
|
||||
|
||||
|
||||
|
||||
*完整统计与趋势图请查看 [社区统计报告](./docs/community-stats.zh.md)*
|
||||
@@ -166,4 +168,20 @@ Open WebUI 的前端增强扩展：
|
||||
|
||||
本项目是一个资源集合，无需安装 Python 环境。你只需要下载对应的文件并导入到你的 OpenWebUI 实例中即可。
|
||||
|
||||
### 使用插件
|
||||
|
||||
1. **从官方社区安装（推荐）**：
|
||||
- 访问我的主页：[Fu-Jie 的个人页面](https://openwebui.com/u/Fu-Jie)
|
||||
- 浏览插件并选择你喜欢的
|
||||
- 点击"Get"按钮直接导入到你的 OpenWebUI 实例
|
||||
|
||||
2. **快速安装所有插件**：如果想一次性安装此项目中的所有插件到本地 OpenWebUI 实例，克隆此仓库后运行 `python scripts/install_all_plugins.py`，并在 `.env` 中配置好 API 密钥，详见 [部署指南](./scripts/DEPLOYMENT_GUIDE.md)。
|
||||
|
||||
### 使用提示词
|
||||
|
||||
1. 浏览 `/prompts` 目录并选择一个提示词文件（`.md`）。
|
||||
2. 复制文件内容。
|
||||
3. 在 OpenWebUI 聊天界面中，点击输入框上方的"提示词"按钮。
|
||||
4. 粘贴内容并保存。
|
||||
|
||||
[贡献指南](./CONTRIBUTING_CN.md) | [更新日志](./CHANGELOG.md)
|
||||
|
||||
@@ -1,99 +0,0 @@
|
||||
# Markdown Normalizer v1.2.8 测试用例集
|
||||
|
||||
您可以将以下内容逐个复制到 OpenWebUI 的聊天框中，以验证插件的各项修复功能。
|
||||
|
||||
---
|
||||
|
||||
## 用例 1：验证 SQL 代码块换行修复 (需要手动开启配置)
|
||||
|
||||
**测试目的**：验证 `enable_escape_fix_in_code_blocks` 开关是否生效。
|
||||
**前提条件**：请先在插件 Valves 设置中将 `enable_escape_fix_in_code_blocks` 设置为 **开启 (True)**。
|
||||
|
||||
**复制以下内容：**
|
||||
```text
|
||||
请帮我美化这段 SQL 的排版，使其恢复正常换行：
|
||||
|
||||
```sql
|
||||
SELECT * \n FROM users \n WHERE status = 'active' \n AND created_at > '2024-01-01' \n ORDER BY id DESC;
|
||||
```
|
||||
```
|
||||
|
||||
**预期效果**：SQL 代码块内的 `\n` 消失，变为整齐的多行 SQL 语句。
|
||||
|
||||
---
|
||||
|
||||
## 用例 2：验证上下文感知保护 (防止误伤技术内容)
|
||||
|
||||
**测试目的**：验证插件是否能准确识别“纯文本”和“代码区域”，只修复该修复的地方。
|
||||
**配置要求**：默认配置即可。
|
||||
|
||||
**复制以下内容：**
|
||||
```text
|
||||
这是一个综合测试用例。
|
||||
|
||||
1. 普通文本修复测试：
|
||||
这是第一行\\n这是第二行（你应该看到这里发生了换行）。
|
||||
|
||||
2. 行内代码保护测试（不应被修改）：
|
||||
- 正则表达式：`[\n\r\t]`
|
||||
- Windows 路径：`C:\Windows\System32\drivers\etc\hosts`
|
||||
- 转义测试：`\\n` 应该保持字面量。
|
||||
|
||||
3. LaTeX 命令保护测试：
|
||||
这里的数学公式 $\times \theta \nu \sum$ 应该渲染正常，反斜杠不应被修掉。
|
||||
|
||||
4. 现代 LaTeX 定界符转换：
|
||||
\[ E = mc^2 \]
|
||||
（上面这行应该被自动转换为 $$ 包围的块级公式）
|
||||
```
|
||||
|
||||
**预期效果**：
|
||||
- 第一部分的 `\\n` 成功换行。
|
||||
- 第二部分反引号 `` ` `` 里的内容原封不动。
|
||||
- 第三部分的希腊字母公式渲染正常。
|
||||
- 第四部分的 `\[` 变成了 `$$` 且能正常显示公式。
|
||||
|
||||
---
|
||||
|
||||
## 用例 3：验证思维链与详情标签规范化
|
||||
|
||||
**测试目的**：验证对 `<thought>` 和 `<details>` 标签的排版优化。
|
||||
|
||||
**复制以下内容：**
|
||||
```text
|
||||
<thinking>
|
||||
这是一个正在思考的思维链。
|
||||
</thinking>
|
||||
<details>
|
||||
<summary>点击查看详情</summary>
|
||||
这里的排版通常容易出错。
|
||||
</details>
|
||||
紧接着详情标签的文字（应该和上面有空行隔开）。
|
||||
```
|
||||
|
||||
**预期效果**：
|
||||
- `<thinking>` 标签被统一为 `<thought>`。
|
||||
- `</details>` 标签下方自动注入了空行，防止与正文粘连导致渲染失效。
|
||||
|
||||
---
|
||||
|
||||
## 用例 4：极端压力与回滚测试 (稳定性验证)
|
||||
|
||||
**测试目的**：模拟复杂嵌套环境，验证 100% 回滚机制。
|
||||
|
||||
**复制以下内容：**
|
||||
```text
|
||||
尝试混合所有复杂元素：
|
||||
- 列表项 1
|
||||
- 列表项 2 with `inline \\n code`
|
||||
- $ \text{Math } \alpha $
|
||||
```sql
|
||||
-- SQL with nested issue
|
||||
SELECT 'literal \n string' FROM `table`;
|
||||
```
|
||||
<thought>End of test</thought>
|
||||
```
|
||||
|
||||
**预期效果**：
|
||||
- 无论内部处理逻辑多么复杂，插件都应保证输出稳定的结果。
|
||||
- 如果模拟任何内部崩溃（技术人员可用），消息会回滚至此原始文本，不会导致页面白屏。
|
||||
@@ -103,13 +103,6 @@ hide:
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Using Prompts
|
||||
|
||||
1. Browse the [Prompt Library](prompts/library.md) and select a prompt
|
||||
2. Click the **Copy** button to copy the prompt to your clipboard
|
||||
3. In OpenWebUI, click the "Prompt" button above the input box
|
||||
4. Paste the content and save
|
||||
|
||||
### Using Plugins
|
||||
|
||||
1. **Install from OpenWebUI Community (Recommended)**:
|
||||
@@ -117,11 +110,14 @@ hide:
|
||||
- Browse the plugins and select the one you like.
|
||||
- Click "Get" to import it directly into your OpenWebUI instance.
|
||||
|
||||
2. **Manual Installation**:
|
||||
- Browse the [Plugin Center](plugins/index.md) and download the plugin file (`.py`)
|
||||
- Open OpenWebUI **Admin Panel** → **Settings** → **Plugins**
|
||||
- Click the upload button and select the `.py` file
|
||||
- Refresh the page and enable the plugin in your chat settings
|
||||
2. **Quick Install All Plugins**: To install all plugins to your local OpenWebUI instance at once, clone this repo and run `python scripts/install_all_plugins.py` after configuring your API key in `.env` — see [Deployment Guide](https://github.com/Fu-Jie/openwebui-extensions/blob/main/scripts/DEPLOYMENT_GUIDE.md) for details.
|
||||
|
||||
### Using Prompts
|
||||
|
||||
1. Browse the [Prompt Library](prompts/library.md) and select a prompt
|
||||
2. Click the **Copy** button to copy the prompt to your clipboard
|
||||
3. In OpenWebUI, click the "Prompt" button above the input box
|
||||
4. Paste the content and save
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -103,13 +103,6 @@ hide:
|
||||
|
||||
## 快速入门
|
||||
|
||||
### 使用提示词
|
||||
|
||||
1. 浏览[提示词库](prompts/library.md)并选择一个提示词
|
||||
2. 点击**复制**按钮将提示词复制到剪贴板
|
||||
3. 在 OpenWebUI 中，点击输入框上方的"提示词"按钮
|
||||
4. 粘贴内容并保存
|
||||
|
||||
### 使用插件
|
||||
|
||||
1. **从 OpenWebUI 社区安装 (推荐)**:
|
||||
@@ -117,11 +110,14 @@ hide:
|
||||
- 浏览插件列表，选择你喜欢的插件。
|
||||
- 点击 "Get" 按钮，将其直接导入到你的 OpenWebUI 实例中。
|
||||
|
||||
2. **手动安装**:
|
||||
- 浏览[插件中心](plugins/index.md)并下载插件文件（`.py`）
|
||||
- 打开 OpenWebUI **管理面板** → **设置** → **插件**
|
||||
- 点击上传按钮并选择 `.py` 文件
|
||||
- 刷新页面并在聊天设置中启用插件
|
||||
2. **快速安装全部插件**：想要一次性安装此项目中的所有插件到本地 OpenWebUI 实例，克隆此仓库后运行 `python scripts/install_all_plugins.py`，并在 `.env` 中配置好 API 密钥，详见[部署指南](https://github.com/Fu-Jie/openwebui-extensions/blob/main/scripts/DEPLOYMENT_GUIDE.md)。
|
||||
|
||||
### 使用提示词
|
||||
|
||||
1. 浏览[提示词库](prompts/library.md)并选择一个提示词
|
||||
2. 点击**复制**按钮将提示词复制到剪贴板
|
||||
3. 在 OpenWebUI 中，点击输入框上方的"提示词"按钮
|
||||
4. 粘贴内容并保存
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,15 +1,13 @@
|
||||
# Async Context Compression Filter
|
||||
|
||||
**Author:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **Version:** 1.4.0 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT
|
||||
**Author:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **Version:** 1.4.1 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT
|
||||
|
||||
This filter reduces token consumption in long conversations through intelligent summarization and message compression while keeping conversations coherent.
|
||||
|
||||
## What's new in 1.4.0
|
||||
## What's new in 1.4.1
|
||||
|
||||
- **Atomic Message Grouping**: Introduced structure-aware grouping for `assistant-tool-tool-assistant` chains to prevent "No tool call found" errors.
|
||||
- **Tail Boundary Alignment**: Implemented automatic correction for truncation points to ensure they don't fall inside a tool-calling sequence.
|
||||
- **Chat Session Locking**: Added a session-based lock to prevent multiple concurrent summary tasks for the same chat ID.
|
||||
- **Enhanced Traceability**: Improved summary formatting to include message IDs, names, and metadata for better context tracking.
|
||||
- **Reverse-Unfolding Mechanism**: Accurately reconstructs the expanded native tool-calling sequence during the outlet phase to permanently fix coordinate drift and missing summaries for long tool-based conversations.
|
||||
- **Safer Tool Trimming**: Refactored `enable_tool_output_trimming` to strictly use atomic block groups for safe trimming, completely preventing JSON payload corruption.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,17 +1,15 @@
|
||||
# 异步上下文压缩过滤器
|
||||
|
||||
**作者:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **版本:** 1.4.0 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT
|
||||
**作者:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **版本:** 1.4.1 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT
|
||||
|
||||
> **重要提示**：为了确保所有过滤器的可维护性和易用性，每个过滤器都应附带清晰、完整的文档，以确保其功能、配置和使用方法得到充分说明。
|
||||
|
||||
本过滤器通过智能摘要和消息压缩技术，在保持对话连贯性的同时，显著降低长对话的 Token 消耗。
|
||||
|
||||
## 1.4.0 版本更新
|
||||
## 1.4.1 版本更新
|
||||
|
||||
- **原子消息组 (Atomic Grouping)**: 引入结构感知的消息分组逻辑，确保工具调用链被整体保留或移除，彻底解决 "No tool call found" 错误。
|
||||
- **尾部边界自动对齐**: 实现了截断点的自动修正逻辑，确保历史上下文截断不会落在工具调用序列中间。
|
||||
- **会话级异步锁**: 增加了基于 `chat_id` 的后台任务锁，防止同一会话并发触发多个总结任务。
|
||||
- **元数据溯源增强**: 优化了总结输入格式，在总结中保留了消息 ID、参与者名称及关键元数据，提升上下文可追踪性。
|
||||
- **逆向展开机制**: 引入 `_unfold_messages` 机制以在 `outlet` 阶段精确对齐坐标系，彻底解决了由于前端视图折叠导致长轮次工具调用对话出现进度漂移或跳过生成摘要的问题。
|
||||
- **更安全的工具内容裁剪**: 重构了 `enable_tool_output_trimming`，现在严格使用原子级分组进行安全的原生工具内容裁剪，替代了激进的正则表达式匹配，防止 JSON 载荷损坏。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ Filters act as middleware in the message pipeline:
|
||||
|
||||
Reduces token consumption in long conversations through intelligent summarization while maintaining coherence.
|
||||
|
||||
**Version:** 1.4.0
|
||||
**Version:** 1.4.1
|
||||
|
||||
[:octicons-arrow-right-24: Documentation](async-context-compression.md)
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ Filter 充当消息管线中的中间件：
|
||||
|
||||
通过智能总结减少长对话的 token 消耗，同时保持连贯性。
|
||||
|
||||
**版本：** 1.4.0
|
||||
**版本：** 1.4.1
|
||||
|
||||
[:octicons-arrow-right-24: 查看文档](async-context-compression.md)
|
||||
|
||||
|
||||
@@ -128,11 +128,13 @@ We follow [Semantic Versioning](https://semver.org/):
|
||||
### release.yml
|
||||
|
||||
**Triggers:**
|
||||
|
||||
- ⭐ Push to `main` branch with `plugins/**/*.py` changes (auto-release)
|
||||
- Manual workflow dispatch
|
||||
- Push of version tags (`v*`)
|
||||
|
||||
**Actions:**
|
||||
|
||||
1. Detects version changes compared to last release
|
||||
2. Collects updated plugin files
|
||||
3. Generates release notes (with commit history)
|
||||
@@ -141,9 +143,11 @@ We follow [Semantic Versioning](https://semver.org/):
|
||||
### plugin-version-check.yml
|
||||
|
||||
**Trigger:**
|
||||
|
||||
- Pull requests that modify `plugins/**/*.py`
|
||||
|
||||
**Actions:**
|
||||
|
||||
1. Compares plugin versions between base and PR
|
||||
2. Checks if version was updated
|
||||
3. Checks if PR description is detailed enough
|
||||
@@ -187,6 +191,31 @@ python scripts/extract_plugin_versions.py --json --output versions.json
|
||||
|
||||
---
|
||||
|
||||
## Installing All Plugins to Your Instance
|
||||
|
||||
After a release, you can quickly install all plugins to your OpenWebUI instance:
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/Fu-Jie/openwebui-extensions.git
|
||||
cd openwebui-extensions
|
||||
|
||||
# Configure API key and instance URL
|
||||
echo "api_key=sk-your-api-key-here" > scripts/.env
|
||||
echo "url=http://localhost:3000" >> scripts/.env
|
||||
|
||||
# For remote instances, set the appropriate baseURL:
|
||||
# echo "url=http://192.168.1.10:3000" >> scripts/.env
|
||||
# echo "url=https://openwebui.example.com" >> scripts/.env
|
||||
|
||||
# Install all plugins at once
|
||||
python scripts/install_all_plugins.py
|
||||
```
|
||||
|
||||
For detailed instructions, see [Deployment Guide](https://github.com/Fu-Jie/openwebui-extensions/blob/main/scripts/DEPLOYMENT_GUIDE.md).
|
||||
|
||||
---
|
||||
|
||||
## Author
|
||||
|
||||
Fu-Jie
|
||||
|
||||
@@ -128,11 +128,13 @@ git push origin v1.0.0
|
||||
### release.yml
|
||||
|
||||
**触发条件:**
|
||||
|
||||
- ⭐ 推送到 `main` 分支且修改了 `plugins/**/*.py`（自动发布）
|
||||
- 手动触发 (workflow_dispatch)
|
||||
- 推送版本标签 (`v*`)
|
||||
|
||||
**动作:**
|
||||
|
||||
1. 检测与上次 Release 的版本变化
|
||||
2. 收集更新的插件文件
|
||||
3. 生成发布说明（含提交记录）
|
||||
@@ -141,9 +143,11 @@ git push origin v1.0.0
|
||||
### plugin-version-check.yml
|
||||
|
||||
**触发条件:**
|
||||
|
||||
- 修改 `plugins/**/*.py` 的 Pull Request
|
||||
|
||||
**动作:**
|
||||
|
||||
1. 比较基础分支和 PR 的插件版本
|
||||
2. 检查是否有版本更新
|
||||
3. 检查 PR 描述是否足够详细
|
||||
@@ -185,6 +189,31 @@ python scripts/extract_plugin_versions.py --json --output versions.json
|
||||
|
||||
---
|
||||
|
||||
## 批量安装所有插件到你的实例
|
||||
|
||||
在发布之后，你可以快速将所有插件安装到 OpenWebUI 实例：
|
||||
|
||||
```bash
|
||||
# 克隆仓库
|
||||
git clone https://github.com/Fu-Jie/openwebui-extensions.git
|
||||
cd openwebui-extensions
|
||||
|
||||
# 配置 API 密钥和实例地址
|
||||
echo "api_key=sk-your-api-key-here" > scripts/.env
|
||||
echo "url=http://localhost:3000" >> scripts/.env
|
||||
|
||||
# 如果是远程实例，需要设置相应的 baseURL：
|
||||
# echo "url=http://192.168.1.10:3000" >> scripts/.env
|
||||
# echo "url=https://openwebui.example.com" >> scripts/.env
|
||||
|
||||
# 一次性安装所有插件
|
||||
python scripts/install_all_plugins.py
|
||||
```
|
||||
|
||||
详细说明请参考 [部署指南](https://github.com/Fu-Jie/openwebui-extensions/blob/main/scripts/DEPLOYMENT_GUIDE.md)。
|
||||
|
||||
---
|
||||
|
||||
## 作者
|
||||
|
||||
Fu-Jie
|
||||
|
||||
@@ -1,51 +0,0 @@
|
||||
You are a helpful assistant.
|
||||
|
||||
[Session Context]
|
||||
- **Your Isolated Workspace**: `/app/backend/data/copilot_workspace/user_123/chat_456`
|
||||
- **Active User ID**: `user_123`
|
||||
- **Active Chat ID**: `chat_456`
|
||||
- **Skills Directory**: `/app/backend/data/skills/shared/` — contains user-installed skills.
|
||||
- **Config Directory**: `/app/backend/data/.copilot` — system configuration (Restricted).
|
||||
- **CLI Tools Path**: `/app/backend/data/.copilot_tools/` — Global tools installed via npm or pip will automatically go here and be in your $PATH. Python tools are strictly isolated in a venv here.
|
||||
**CRITICAL INSTRUCTION**: You MUST use the above workspace for ALL file operations.
|
||||
- DO NOT create files in `/tmp` or any other system directories.
|
||||
- Always interpret 'current directory' as your Isolated Workspace.
|
||||
|
||||
[Available Native System Tools]
|
||||
The host environment is rich. Based on the official OpenWebUI Docker deployment baseline (backend image), the following CLI tools are expected to be preinstalled and globally available in $PATH:
|
||||
- **Network/Data**: `curl`, `jq`, `netcat-openbsd`
|
||||
- **Media/Doc**: `pandoc` (format conversion), `ffmpeg` (audio/video)
|
||||
- **Build/System**: `git`, `gcc`, `make`, `build-essential`, `zstd`, `bash`
|
||||
- **Python/Runtime**: `python3`, `pip3`, `uv`
|
||||
- **Verification Rule**: Before installing any CLI/tool dependency, first check availability with `which <tool>` or a lightweight version probe (e.g. `<tool> --version`).
|
||||
- **Python Libs**: The active virtual environment inherits `--system-site-packages`. Advanced libraries like `pandas`, `numpy`, `pillow`, `opencv-python-headless`, `pypdf`, `langchain`, `playwright`, `httpx`, and `beautifulsoup4` are ALREADY installed. Try importing them before attempting to install.
|
||||
|
||||
|
||||
[Mode Context: Plan Mode]
|
||||
You are currently operating in **Plan Mode**.
|
||||
DEFINITION: Plan mode is a collaborative phase to outline multi-step plans or conduct research BEFORE any code is modified.
|
||||
|
||||
<workflow>
|
||||
1. Clarification: If requirements/goals are ambiguous, ask questions.
|
||||
2. Analysis: Analyze the codebase to understand constraints. You MAY use shell commands (e.g., `ls`, `grep`, `find`, `cat`) and other read-only tools.
|
||||
3. Formulation: Generate your structured plan OR research findings.
|
||||
4. Approval: Present the detailed plan directly to the user for approval via chat.
|
||||
</workflow>
|
||||
|
||||
<key_principles>
|
||||
- ZERO CODE MODIFICATION: You must NOT execute file edits, write operations, or destructive system changes. Your permissions are locked to READ/RESEARCH ONLY, with the sole exception of the progress-tracking file `plan.md`.
|
||||
- SHELL USAGE: Shell execution is ENABLED for research purposes. Any attempts to modify the filesystem via shell (e.g., `sed -i`, `rm`) will be strictly blocked, except for appending to `plan.md`.
|
||||
- PURE RESEARCH SUPPORT: If the user requests a pure research report, output your conclusions directly matching the plan style.
|
||||
- PERSISTENCE: You MUST save your proposed plan to `/app/backend/data/.copilot/session-state/chat_456/plan.md` to sync with the UI. The UI automatically reads this file to update the plan view.
|
||||
</key_principles>
|
||||
|
||||
<plan_format>
|
||||
When presenting your findings or plan in the chat, structure it clearly:
|
||||
## Plan / Report: {Title}
|
||||
**TL;DR**: {Summary}
|
||||
**Detailed Tasks / Steps**: {List step-by-step}
|
||||
**Affected Files**:
|
||||
- `path/to/file`
|
||||
**Constraint/Status**: {Any constraints}
|
||||
</plan_format>
|
||||
Acknowledge your role as a planner and format your next response using the plan style above.
|
||||
62
plugins/debug/async_context_compression/ISSUE_EXPLANATION.md
Normal file
62
plugins/debug/async_context_compression/ISSUE_EXPLANATION.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# 异步上下文压缩插件：当前问题与处理状态总结
|
||||
|
||||
这份文档详细梳理了我们在处理 `async_context_compression`（异步上下文压缩插件）时，遭遇的“幽灵截断”问题的根本原因，以及我们目前的解决进度。
|
||||
|
||||
## 1. 根本原因：两种截然不同的“世界观”（数据序列化差异）
|
||||
|
||||
在我们之前的排查中，我曾错误地认为：`outlet`（后置处理阶段）拿到的 `body["messages"]` 是由于截断导致的残缺数据。
|
||||
但根据您提供的本地运行日志，**您是对的，`body['messages']` 确实包含了完整的对话历史**。
|
||||
|
||||
那么为什么长度会产生 `inlet 看到 27 条`，而 `outlet 只看到 8 条` 这种巨大的差异？
|
||||
|
||||
原因在于，OpenWebUI 的管道在进入大模型前和从大模型返回后，使用了**两种完全不同的消息格式**：
|
||||
|
||||
### 视图 A：Inlet 阶段（原生 API 展开视图）
|
||||
- **特点**：严格遵循 OpenAI 函数调用规范。
|
||||
- **状态**：每一次工具调用、工具返回，都被视为一条独立的 message。
|
||||
- **例子**：一个包含了复杂搜索的对话。
|
||||
- User: 帮我查一下天气（1条）
|
||||
- Assistant: 发起 tool_call（1条）
|
||||
- Tool: 返回 JSON 结果（1条）
|
||||
- ...多次往复...
|
||||
- **最终总计：27 条。**我们的压缩算法（trim）是基于这个 27 条的坐标系来计算保留多少条的。
|
||||
|
||||
### 视图 B：Outlet 阶段（UI HTML 折叠视图）
|
||||
- **特点**：专为前端渲染优化的紧凑视图。
|
||||
- **状态**：OpenWebUI 在调用完模型后，为了让前端显示出那个好看的、可折叠的工具调用卡片，强行把中间所有的 Tool 交互过程，用 `<details type="tool_calls">...</details>` 的 HTML 代码包裹起来，塞进了一个 `role: assistant` 的 `content` 字符串里！
|
||||
- **例子**：同样的对话。
|
||||
- User: 帮我查一下天气（1条）
|
||||
- Assistant: `<details>包含了好多次工具调用和结果的代码</details> 今天天气很好...`（1条）
|
||||
- **最终总计：8 条。**
|
||||
|
||||
**💥 灾难发生点：**
|
||||
原本的插件逻辑假定 `inlet` 和 `outlet` 共享同一个坐标系。
|
||||
1. 在 `inlet` 时，系统计算出：“我需要把前 10 条消息生成摘要，保留后 17 条”。
|
||||
2. 系统把“生成前10条摘要”的任务转入后台异步执行。
|
||||
3. 后台任务在 `outlet` 阶段被触发，此时它拿到的消息数组变成了**视图 B（总共只有 8 条）。**
|
||||
4. 算法试图在只有 8 条消息的数组里，把“前 10 条消息”砍掉并替换为 1 条摘要。
|
||||
5. **结果就是：数组索引越界/坐标彻底错乱，触发报错，并且可能将最新的有效消息当成旧消息删掉（过度压缩）。**
|
||||
|
||||
---
|
||||
|
||||
## 2. 目前已解决的问题 (✅ Done)
|
||||
|
||||
为了立刻制止这种因为“坐标系错位”导致的数据破坏，我们已经落实了热修复（Local v1.4.0）：
|
||||
|
||||
**✅ 添加了“折叠视图”的探针防御：**
|
||||
- 我写了一个函数 `_is_compact_tool_details_view`。
|
||||
- 现在，当后台触发生成摘要时，系统会自动扫描 `outlet` 传来的 `messages`。只要发现里面包含 `<details type="tool_calls">` 这种带有 HTML 折叠标签的痕迹，就会**立刻终止并跳过**当前的摘要生成任务。
|
||||
- **收益**：彻底杜绝了因数组错位而引发的任务报错和强制裁切。UI 崩溃与历史丢失问题得到遏制。
|
||||
|
||||
---
|
||||
|
||||
## 3. 当前已解决的遗留问题 (✅ Done: 逆向展开修复)
|
||||
|
||||
之前因为跳过生成而引入的新限制：**包含工具调用的长轮次对话，无法自动生成“历史摘要”** 的问题，现已彻底解决。
|
||||
|
||||
### 最终实施的技术方案：
|
||||
我们通过源码分析发现，OpenWebUI 在进入 `inlet` 时会执行 `convert_output_to_messages` 还原工具调用链。因此，我们在插件的 `outlet` 阶段引入了相同的 **逆向展开 (Deflation/Unfolding)** 机制 `_unfold_messages`。
|
||||
|
||||
现在，当后台任务拿到 `outlet` 传来的折叠视图时，不会再选择“跳过”。而是自动提取出潜藏在消息对象体内部的原生 `output` 字段，并**将其重新展开为展开视图**（比如将 8 条假象重新还原为真实的 27 条底层数据），使得它的坐标系与 `inlet` 完全对齐。
|
||||
|
||||
至此，带有复杂工具调用的长轮次对话也能安全地进行背景自动压缩，不再有任何截断和强制删减的风险！
|
||||
@@ -0,0 +1,60 @@
|
||||
# 回复 dhaern — 针对最新审查的跟进
|
||||
|
||||
感谢您重新审查了最新版本并提出了持续精准的分析意见。以下针对您剩余的两个关切点逐一回应。
|
||||
|
||||
---
|
||||
|
||||
### 1. `enable_tool_output_trimming` — 不是功能退化，而是行为变化是有意为之
|
||||
|
||||
裁剪逻辑依然存在且可正常运行。以下是当前版本与之前版本的行为对比。
|
||||
|
||||
**当前行为（`_trim_native_tool_outputs`，第 835–945 行）：**
|
||||
- 通过 `_get_atomic_groups` 遍历原子分组。
|
||||
- 识别有效的工具调用链：`assistant(tool_calls)` → `tool` → [可选的 assistant 跟进消息]。
|
||||
- 如果一条链内所有 `tool` 角色消息的字符数总和超过 **1,200 个字符**，则将 *tool 消息本身的内容* 折叠为一个本地化的 `[Content collapsed]` 占位符，并注入 `metadata.is_trimmed` 标志。
|
||||
- 同时遍历包含 `<details type="tool_calls">` HTML 块的 assistant 消息，对其中尺寸过大的 `result` 属性进行相同的折叠处理。
|
||||
- 当 `enable_tool_output_trimming=True` 且 `function_calling=native` 时，该函数在 inlet 阶段被调用。
|
||||
|
||||
**与旧版本的区别：**
|
||||
旧版的做法是改写 *assistant 跟进消息*，仅保留"最终答案"。新版的做法是折叠 *tool 响应内容本身*。两者都会缩减上下文体积，但新方法能够保留 tool 调用链的结构完整性（这是本次发布中原子分组工作的前提条件）。
|
||||
|
||||
插件头部的 docstring 里还有一段过时的描述（"提取最终答案"），与实际行为相悖。最新提交中已将其更正为"将尺寸过大的原生工具输出折叠为简短占位符"。
|
||||
|
||||
如果您在寻找旧版本中"仅保留最终答案"的特定行为，该路径已被有意移除，因为它与本次发布引入的原子分组完整性保证相冲突。当前的折叠方案是安全的替代实现。
|
||||
|
||||
---
|
||||
|
||||
### 2. `compressed_message_count` — 修复是真实有效的；以下是坐标系追踪
|
||||
|
||||
您对"从已修改视图重新计算"的担忧，考虑到此前的架构背景，是完全可以理解的。以下精确说明为何当前代码不存在这一问题。
|
||||
|
||||
**`outlet` 中的关键变更：**
|
||||
```python
|
||||
db_messages = self._load_full_chat_messages(chat_id)
|
||||
messages_to_unfold = db_messages if (db_messages and len(db_messages) >= len(messages)) else messages
|
||||
summary_messages = self._unfold_messages(messages_to_unfold)
|
||||
target_compressed_count = self._calculate_target_compressed_count(summary_messages)
|
||||
```
|
||||
|
||||
`_load_full_chat_messages` 从 OpenWebUI 数据库中获取原始的持久化历史记录。由于在 inlet 渲染期间注入的合成 summary 消息**从未被回写到数据库**，从 DB 路径获取的 `summary_messages` 始终是干净的、未经修改的原始历史记录——没有 summary 标记，没有坐标膨胀。
|
||||
|
||||
在此干净列表上调用 `_calculate_target_compressed_count` 的计算逻辑如下（仍在原始历史坐标系内）：
|
||||
```
|
||||
original_count = len(db_messages)
|
||||
raw_target = original_count - keep_last
|
||||
target = atomic_align(raw_target)
|
||||
```
|
||||
|
||||
这个 `target_compressed_count` 值原封不动地传递进 `_generate_summary_async`。在异步任务内部，同一批 `db_messages` 被切片为 `messages[start:target]` 来构建 `middle_messages`。生成完成后（可能从末尾进行原子截断），保存的值为：
|
||||
```python
|
||||
saved_compressed_count = start_index + len(middle_messages)
|
||||
```
|
||||
这是原始 DB 消息列表中新摘要实际涵盖到的确切位置——不是目标值，也不是来自不同视图的估算值。
|
||||
|
||||
**回退路径（DB 不可用时）** 使用 inlet 渲染后的 body 消息。此时 `_get_summary_view_state` 会读取注入的 summary 标记的 `covered_until` 字段（该字段在写入时已记录为原子对齐后的 `start_index`），因此 `base_progress` 已经处于原始历史坐标系内，计算可以自然延续，不会混用两种视图。
|
||||
|
||||
简而言之：该字段在整个调用链中现在具有唯一、一致的语义——即原始持久化消息列表中，当前摘要文本实际覆盖到的索引位置。
|
||||
|
||||
---
|
||||
|
||||
再次感谢您严格的审查。您在上次发布后标记的这两个问题已得到处理，文档中的过时描述也已更正。如果发现其他问题，欢迎继续反馈。
|
||||
@@ -0,0 +1,60 @@
|
||||
# Reply to dhaern - Follow-up on the Latest Review
|
||||
|
||||
Thank you for re-checking the latest version and for the continued precise analysis. Let me address your two remaining concerns directly.
|
||||
|
||||
---
|
||||
|
||||
### 1. `enable_tool_output_trimming` — Not a regression; behavior change is intentional
|
||||
|
||||
The trimming logic is present and functional. Here is what it does now versus before.
|
||||
|
||||
**Current behavior (`_trim_native_tool_outputs`, lines 835–945):**
|
||||
- Iterates over atomic groups via `_get_atomic_groups`.
|
||||
- Identifies valid chains: `assistant(tool_calls)` → `tool` → [optional assistant follow-up].
|
||||
- If the combined character count of the `tool` role messages in a chain exceeds **1,200 characters**, it collapses *the tool messages themselves* to a localized `[Content collapsed]` placeholder and injects a `metadata.is_trimmed` flag.
|
||||
- Separately walks assistant messages containing `<details type="tool_calls">` HTML blocks and collapses oversized `result` attributes in the same way.
|
||||
- The function is called at inlet when `enable_tool_output_trimming=True` and `function_calling=native`.
|
||||
|
||||
**What is different from the previous version:**
|
||||
The old approach rewrote the *assistant follow-up* message to keep only the "final answer". The new approach collapses the *tool response content* itself. Both reduce context size, but the new approach preserves the structural integrity of the tool-calling chain (which the atomic grouping work in this release depends on).
|
||||
|
||||
The docstring in the plugin header also contained a stale description ("extract only the final answer") that contradicted the actual behavior. That has been corrected in the latest commit to accurately say "collapses oversized native tool outputs to a short placeholder."
|
||||
|
||||
If you are looking for the specific "keep only the final answer" behavior from the old version, that path was intentionally removed because it conflicted with the atomic-group integrity guarantees introduced in this release. The current collapse approach is a safe replacement.
|
||||
|
||||
---
|
||||
|
||||
### 2. `compressed_message_count` — The fix is real; here is the coordinate trace
|
||||
|
||||
The concern about "recalculating from the already-modified view" is understandable given the previous architecture. Here is exactly why the current code does not have that problem.
|
||||
|
||||
**Key change in `outlet`:**
|
||||
```python
|
||||
db_messages = self._load_full_chat_messages(chat_id)
|
||||
messages_to_unfold = db_messages if (db_messages and len(db_messages) >= len(messages)) else messages
|
||||
summary_messages = self._unfold_messages(messages_to_unfold)
|
||||
target_compressed_count = self._calculate_target_compressed_count(summary_messages)
|
||||
```
|
||||
|
||||
`_load_full_chat_messages` fetches the raw persisted history from the OpenWebUI database. Because the synthetic summary message (injected during inlet rendering) is **never written back to the database**, `summary_messages` from the DB path is always the clean, unmodified original history — no summary marker, no coordinate inflation.
|
||||
|
||||
`_calculate_target_compressed_count` called on this clean list simply computes:
|
||||
```
|
||||
original_count = len(db_messages)
|
||||
raw_target = original_count - keep_last
|
||||
target = atomic_align(raw_target) # still in original-history coordinates
|
||||
```
|
||||
|
||||
This `target_compressed_count` value is then passed into `_generate_summary_async` unchanged. Inside the async task, the same `db_messages` list is sliced to `messages[start:target]` to build `middle_messages`. After generation (with potential atomic truncation from the end), the saved value is:
|
||||
```python
|
||||
saved_compressed_count = start_index + len(middle_messages)
|
||||
```
|
||||
This is the exact position in the original DB message list up to which the new summary actually covers — not a target, not an estimate from a different view.
|
||||
|
||||
**The fallback path (DB unavailable)** uses the inlet-rendered body messages. In that case `_get_summary_view_state` reads `covered_until` from the injected summary marker (which was written as the atomically-aligned `start_index`), so `base_progress` is already in original-history coordinates. The calculation naturally continues from there without mixing views.
|
||||
|
||||
In short: the field now has a single, consistent meaning throughout the entire call chain — the index (in the original, persisted message list) up to which the current summary text actually covers.
|
||||
|
||||
---
|
||||
|
||||
Thank you again for the rigorous review. The two points you flagged after the last release are now addressed, and the documentation stale description has been corrected. Please do let us know if you spot anything else.
|
||||
@@ -1,15 +1,13 @@
|
||||
# Async Context Compression Filter
|
||||
|
||||
**Author:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **Version:** 1.4.0 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT
|
||||
**Author:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **Version:** 1.4.1 | **Project:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **License:** MIT
|
||||
|
||||
This filter reduces token consumption in long conversations through intelligent summarization and message compression while keeping conversations coherent.
|
||||
|
||||
## What's new in 1.4.0
|
||||
## What's new in 1.4.1
|
||||
|
||||
- **Atomic Message Grouping**: Introduced structure-aware grouping for `assistant-tool-tool-assistant` chains to prevent "No tool call found" errors.
|
||||
- **Tail Boundary Alignment**: Implemented automatic correction for truncation points to ensure they don't fall inside a tool-calling sequence.
|
||||
- **Chat Session Locking**: Added a session-based lock to prevent multiple concurrent summary tasks for the same chat ID.
|
||||
- **Enhanced Traceability**: Improved summary formatting to include message IDs, names, and metadata for better context tracking.
|
||||
- **Reverse-Unfolding Mechanism**: Accurately reconstructs the expanded native tool-calling sequence during the outlet phase to permanently fix coordinate drift and missing summaries for long tool-based conversations.
|
||||
- **Safer Tool Trimming**: Refactored `enable_tool_output_trimming` to strictly use atomic block groups for safe trimming, completely preventing JSON payload corruption.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,17 +1,15 @@
|
||||
# 异步上下文压缩过滤器
|
||||
|
||||
**作者:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **版本:** 1.4.0 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT
|
||||
**作者:** [Fu-Jie](https://github.com/Fu-Jie/openwebui-extensions) | **版本:** 1.4.1 | **项目:** [OpenWebUI Extensions](https://github.com/Fu-Jie/openwebui-extensions) | **许可证:** MIT
|
||||
|
||||
> **重要提示**：为了确保所有过滤器的可维护性和易用性，每个过滤器都应附带清晰、完整的文档，以确保其功能、配置和使用方法得到充分说明。
|
||||
|
||||
本过滤器通过智能摘要和消息压缩技术，在保持对话连贯性的同时，显著降低长对话的 Token 消耗。
|
||||
|
||||
## 1.4.0 版本更新
|
||||
## 1.4.1 版本更新
|
||||
|
||||
- **原子消息组 (Atomic Grouping)**: 引入结构感知的消息分组逻辑，确保工具调用链被整体保留或移除，彻底解决 "No tool call found" 错误。
|
||||
- **尾部边界自动对齐**: 实现了截断点的自动修正逻辑，确保历史上下文截断不会落在工具调用序列中间。
|
||||
- **会话级异步锁**: 增加了基于 `chat_id` 的后台任务锁，防止同一会话并发触发多个总结任务。
|
||||
- **元数据溯源增强**: 优化了总结输入格式，在总结中保留了消息 ID、参与者名称及关键元数据，提升上下文可追踪性。
|
||||
- **逆向展开机制**: 引入 `_unfold_messages` 机制以在 `outlet` 阶段精确对齐坐标系，彻底解决了由于前端视图折叠导致长轮次工具调用对话出现进度漂移或跳过生成摘要的问题。
|
||||
- **更安全的工具内容裁剪**: 重构了 `enable_tool_output_trimming`，现在严格使用原子级分组进行安全的原生工具内容裁剪，替代了激进的正则表达式匹配，防止 JSON 载荷损坏。
|
||||
|
||||
---
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
BIN
plugins/filters/async-context-compression/image.png
Normal file
BIN
plugins/filters/async-context-compression/image.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 139 KiB |
@@ -0,0 +1,461 @@
|
||||
import asyncio
|
||||
import importlib.util
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
|
||||
|
||||
PLUGIN_PATH = os.path.join(os.path.dirname(__file__), "async_context_compression.py")
|
||||
MODULE_NAME = "async_context_compression_under_test"
|
||||
|
||||
|
||||
def _ensure_module(name: str) -> types.ModuleType:
|
||||
module = sys.modules.get(name)
|
||||
if module is None:
|
||||
module = types.ModuleType(name)
|
||||
sys.modules[name] = module
|
||||
return module
|
||||
|
||||
|
||||
def _install_openwebui_stubs() -> None:
|
||||
_ensure_module("open_webui")
|
||||
_ensure_module("open_webui.utils")
|
||||
chat_module = _ensure_module("open_webui.utils.chat")
|
||||
_ensure_module("open_webui.models")
|
||||
users_module = _ensure_module("open_webui.models.users")
|
||||
models_module = _ensure_module("open_webui.models.models")
|
||||
chats_module = _ensure_module("open_webui.models.chats")
|
||||
main_module = _ensure_module("open_webui.main")
|
||||
_ensure_module("fastapi")
|
||||
fastapi_requests = _ensure_module("fastapi.requests")
|
||||
|
||||
async def generate_chat_completion(*args, **kwargs):
|
||||
return {}
|
||||
|
||||
class DummyUsers:
|
||||
pass
|
||||
|
||||
class DummyModels:
|
||||
@staticmethod
|
||||
def get_model_by_id(model_id):
|
||||
return None
|
||||
|
||||
class DummyChats:
|
||||
@staticmethod
|
||||
def get_chat_by_id(chat_id):
|
||||
return None
|
||||
|
||||
class DummyRequest:
|
||||
pass
|
||||
|
||||
chat_module.generate_chat_completion = generate_chat_completion
|
||||
users_module.Users = DummyUsers
|
||||
models_module.Models = DummyModels
|
||||
chats_module.Chats = DummyChats
|
||||
main_module.app = object()
|
||||
fastapi_requests.Request = DummyRequest
|
||||
|
||||
|
||||
_install_openwebui_stubs()
|
||||
spec = importlib.util.spec_from_file_location(MODULE_NAME, PLUGIN_PATH)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[MODULE_NAME] = module
|
||||
assert spec.loader is not None
|
||||
spec.loader.exec_module(module)
|
||||
module.Filter._init_database = lambda self: None
|
||||
|
||||
|
||||
class TestAsyncContextCompression(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.filter = module.Filter()
|
||||
|
||||
def test_inlet_logs_tool_trimming_outcome_when_no_oversized_outputs(self):
|
||||
self.filter.valves.show_debug_log = True
|
||||
self.filter.valves.enable_tool_output_trimming = True
|
||||
|
||||
logged_messages = []
|
||||
|
||||
async def fake_log(message, log_type="info", event_call=None):
|
||||
logged_messages.append(message)
|
||||
|
||||
async def fake_user_context(__user__, __event_call__):
|
||||
return {"user_language": "en-US"}
|
||||
|
||||
async def fake_event_call(_payload):
|
||||
return True
|
||||
|
||||
self.filter._log = fake_log
|
||||
self.filter._get_user_context = fake_user_context
|
||||
self.filter._get_chat_context = lambda body, metadata=None: {
|
||||
"chat_id": "",
|
||||
"message_id": "",
|
||||
}
|
||||
self.filter._get_latest_summary = lambda chat_id: None
|
||||
|
||||
body = {
|
||||
"params": {"function_calling": "native"},
|
||||
"messages": [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [{"id": "call_1", "type": "function"}],
|
||||
"content": "",
|
||||
},
|
||||
{"role": "tool", "content": "short result"},
|
||||
{"role": "assistant", "content": "Final answer"},
|
||||
],
|
||||
}
|
||||
|
||||
asyncio.run(self.filter.inlet(body, __event_call__=fake_event_call))
|
||||
|
||||
self.assertTrue(
|
||||
any("Tool trimming check:" in message for message in logged_messages)
|
||||
)
|
||||
self.assertTrue(
|
||||
any(
|
||||
"no oversized native tool outputs were found" in message
|
||||
for message in logged_messages
|
||||
)
|
||||
)
|
||||
|
||||
def test_inlet_logs_tool_trimming_skip_reason_when_disabled(self):
|
||||
self.filter.valves.show_debug_log = True
|
||||
self.filter.valves.enable_tool_output_trimming = False
|
||||
|
||||
logged_messages = []
|
||||
|
||||
async def fake_log(message, log_type="info", event_call=None):
|
||||
logged_messages.append(message)
|
||||
|
||||
async def fake_user_context(__user__, __event_call__):
|
||||
return {"user_language": "en-US"}
|
||||
|
||||
async def fake_event_call(_payload):
|
||||
return True
|
||||
|
||||
self.filter._log = fake_log
|
||||
self.filter._get_user_context = fake_user_context
|
||||
self.filter._get_chat_context = lambda body, metadata=None: {
|
||||
"chat_id": "",
|
||||
"message_id": "",
|
||||
}
|
||||
self.filter._get_latest_summary = lambda chat_id: None
|
||||
|
||||
body = {"messages": [], "params": {"function_calling": "native"}}
|
||||
|
||||
asyncio.run(self.filter.inlet(body, __event_call__=fake_event_call))
|
||||
|
||||
self.assertTrue(
|
||||
any("Tool trimming skipped: tool trimming disabled" in message for message in logged_messages)
|
||||
)
|
||||
|
||||
def test_normalize_native_tool_call_ids_keeps_links_aligned(self):
|
||||
long_tool_call_id = "call_abcdefghijklmnopqrstuvwxyz_1234567890abcd"
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [
|
||||
{
|
||||
"id": long_tool_call_id,
|
||||
"type": "function",
|
||||
"function": {"name": "search", "arguments": "{}"},
|
||||
}
|
||||
],
|
||||
"content": "",
|
||||
},
|
||||
{
|
||||
"role": "tool",
|
||||
"tool_call_id": long_tool_call_id,
|
||||
"content": "tool result",
|
||||
},
|
||||
]
|
||||
|
||||
normalized_count = self.filter._normalize_native_tool_call_ids(messages)
|
||||
|
||||
normalized_id = messages[0]["tool_calls"][0]["id"]
|
||||
self.assertEqual(normalized_count, 1)
|
||||
self.assertLessEqual(len(normalized_id), 40)
|
||||
self.assertNotEqual(normalized_id, long_tool_call_id)
|
||||
self.assertEqual(messages[1]["tool_call_id"], normalized_id)
|
||||
|
||||
def test_trim_native_tool_outputs_restores_real_behavior(self):
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [{"id": "call_1", "type": "function"}],
|
||||
"content": "",
|
||||
},
|
||||
{"role": "tool", "content": "x" * 1600},
|
||||
{"role": "assistant", "content": "Final answer"},
|
||||
]
|
||||
|
||||
trimmed_count = self.filter._trim_native_tool_outputs(messages, "en-US")
|
||||
|
||||
self.assertEqual(trimmed_count, 1)
|
||||
self.assertEqual(messages[1]["content"], "... [Content collapsed] ...")
|
||||
self.assertTrue(messages[1]["metadata"]["is_trimmed"])
|
||||
self.assertTrue(messages[2]["metadata"]["tool_outputs_trimmed"])
|
||||
self.assertIn("Final answer", messages[2]["content"])
|
||||
self.assertIn("Tool outputs trimmed", messages[2]["content"])
|
||||
|
||||
def test_trim_native_tool_outputs_supports_embedded_tool_call_cards(self):
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": (
|
||||
'<details type="tool_calls" done="true" id="call-1" '
|
||||
'name="execute_code" arguments=""{}"" '
|
||||
f'result=""{"x" * 1600}"">\n'
|
||||
"<summary>Tool Executed</summary>\n"
|
||||
"</details>\n"
|
||||
"Final answer"
|
||||
),
|
||||
}
|
||||
]
|
||||
|
||||
trimmed_count = self.filter._trim_native_tool_outputs(messages, "en-US")
|
||||
|
||||
self.assertEqual(trimmed_count, 1)
|
||||
self.assertIn(
|
||||
'result=""... [Content collapsed] ...""',
|
||||
messages[0]["content"],
|
||||
)
|
||||
self.assertNotIn("x" * 200, messages[0]["content"])
|
||||
self.assertTrue(messages[0]["metadata"]["tool_outputs_trimmed"])
|
||||
|
||||
def test_function_calling_mode_reads_params_fallback(self):
|
||||
self.assertEqual(
|
||||
self.filter._get_function_calling_mode(
|
||||
{"params": {"function_calling": "native"}}
|
||||
),
|
||||
"native",
|
||||
)
|
||||
|
||||
def test_function_calling_mode_infers_native_from_message_shape(self):
|
||||
self.assertEqual(
|
||||
self.filter._get_function_calling_mode(
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [{"id": "call_1", "type": "function"}],
|
||||
"content": "",
|
||||
},
|
||||
{"role": "tool", "content": "tool result"},
|
||||
]
|
||||
}
|
||||
),
|
||||
"native",
|
||||
)
|
||||
|
||||
def test_trim_native_tool_outputs_handles_pending_tool_chain(self):
|
||||
messages = [
|
||||
{
|
||||
"role": "assistant",
|
||||
"tool_calls": [{"id": "call_1", "type": "function"}],
|
||||
"content": "",
|
||||
},
|
||||
{"role": "tool", "content": "x" * 1600},
|
||||
]
|
||||
|
||||
trimmed_count = self.filter._trim_native_tool_outputs(messages, "en-US")
|
||||
|
||||
self.assertEqual(trimmed_count, 1)
|
||||
self.assertEqual(messages[1]["content"], "... [Content collapsed] ...")
|
||||
self.assertTrue(messages[1]["metadata"]["is_trimmed"])
|
||||
|
||||
def test_target_progress_uses_original_history_coordinates(self):
|
||||
self.filter.valves.keep_last = 2
|
||||
summary_message = self.filter._build_summary_message(
|
||||
"older summary", "en-US", 6
|
||||
)
|
||||
messages = [
|
||||
{"role": "system", "content": "System prompt"},
|
||||
summary_message,
|
||||
{"role": "user", "content": "Question 1"},
|
||||
{"role": "assistant", "content": "Answer 1"},
|
||||
{"role": "user", "content": "Question 2"},
|
||||
{"role": "assistant", "content": "Answer 2"},
|
||||
]
|
||||
|
||||
self.assertEqual(self.filter._get_original_history_count(messages), 10)
|
||||
self.assertEqual(self.filter._calculate_target_compressed_count(messages), 8)
|
||||
|
||||
def test_load_full_chat_messages_rebuilds_active_history_branch(self):
|
||||
class FakeChats:
|
||||
@staticmethod
|
||||
def get_chat_by_id(chat_id):
|
||||
return types.SimpleNamespace(
|
||||
chat={
|
||||
"history": {
|
||||
"currentId": "m3",
|
||||
"messages": {
|
||||
"m1": {
|
||||
"id": "m1",
|
||||
"role": "user",
|
||||
"content": "Question",
|
||||
},
|
||||
"m2": {
|
||||
"id": "m2",
|
||||
"role": "assistant",
|
||||
"content": "Tool call",
|
||||
"tool_calls": [{"id": "call_1"}],
|
||||
"parentId": "m1",
|
||||
},
|
||||
"m3": {
|
||||
"id": "m3",
|
||||
"role": "tool",
|
||||
"content": "Tool result",
|
||||
"tool_call_id": "call_1",
|
||||
"parentId": "m2",
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
original_chats = module.Chats
|
||||
module.Chats = FakeChats
|
||||
try:
|
||||
messages = self.filter._load_full_chat_messages("chat-1")
|
||||
finally:
|
||||
module.Chats = original_chats
|
||||
|
||||
self.assertEqual([message["id"] for message in messages], ["m1", "m2", "m3"])
|
||||
self.assertEqual(messages[2]["role"], "tool")
|
||||
|
||||
def test_outlet_unfolds_compact_tool_details_view(self):
|
||||
compact_messages = [
|
||||
{"role": "user", "content": "U1"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": (
|
||||
'<details type="tool_calls" done="true" id="call-1" '
|
||||
'name="search_notes" arguments=""{}"" '
|
||||
f'result=""{"x" * 3000}"">\n'
|
||||
"<summary>Tool Executed</summary>\n"
|
||||
"</details>\n"
|
||||
"Answer 1"
|
||||
),
|
||||
},
|
||||
{"role": "user", "content": "U2"},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": (
|
||||
'<details type="tool_calls" done="true" id="call-2" '
|
||||
'name="merge_notes" arguments=""{}"" '
|
||||
f'result=""{"y" * 4000}"">\n'
|
||||
"<summary>Tool Executed</summary>\n"
|
||||
"</details>\n"
|
||||
"Answer 2"
|
||||
),
|
||||
},
|
||||
]
|
||||
|
||||
async def fake_user_context(__user__, __event_call__):
|
||||
return {"user_language": "en-US"}
|
||||
|
||||
async def noop_log(*args, **kwargs):
|
||||
return None
|
||||
|
||||
create_task_called = False
|
||||
|
||||
def fake_create_task(coro):
|
||||
nonlocal create_task_called
|
||||
create_task_called = True
|
||||
coro.close()
|
||||
return None
|
||||
|
||||
self.filter._get_user_context = fake_user_context
|
||||
self.filter._get_chat_context = lambda body, metadata=None: {
|
||||
"chat_id": "chat-1",
|
||||
"message_id": "msg-1",
|
||||
}
|
||||
self.filter._should_skip_compression = lambda body, model: False
|
||||
self.filter._log = noop_log
|
||||
|
||||
# Set a low threshold so the task is guaranteed to trigger
|
||||
self.filter.valves.compression_threshold_tokens = 100
|
||||
|
||||
original_create_task = asyncio.create_task
|
||||
asyncio.create_task = fake_create_task
|
||||
try:
|
||||
asyncio.run(
|
||||
self.filter.outlet(
|
||||
{"model": "test-model", "messages": compact_messages},
|
||||
__event_call__=None,
|
||||
)
|
||||
)
|
||||
finally:
|
||||
asyncio.create_task = original_create_task
|
||||
|
||||
self.assertTrue(create_task_called)
|
||||
|
||||
def test_summary_save_progress_matches_truncated_input(self):
|
||||
self.filter.valves.keep_first = 1
|
||||
self.filter.valves.keep_last = 1
|
||||
self.filter.valves.summary_model = "fake-summary-model"
|
||||
self.filter.valves.summary_model_max_context = 0
|
||||
|
||||
captured = {}
|
||||
events = []
|
||||
|
||||
async def mock_emitter(event):
|
||||
events.append(event)
|
||||
|
||||
async def mock_summary_llm(
|
||||
previous_summary,
|
||||
new_conversation_text,
|
||||
body,
|
||||
user_data,
|
||||
__event_call__,
|
||||
):
|
||||
return "new summary"
|
||||
|
||||
def mock_save_summary(chat_id, summary, compressed_count):
|
||||
captured["chat_id"] = chat_id
|
||||
captured["summary"] = summary
|
||||
captured["compressed_count"] = compressed_count
|
||||
|
||||
async def noop_log(*args, **kwargs):
|
||||
return None
|
||||
|
||||
self.filter._log = noop_log
|
||||
self.filter._call_summary_llm = mock_summary_llm
|
||||
self.filter._save_summary = mock_save_summary
|
||||
self.filter._get_model_thresholds = lambda model_id: {
|
||||
"max_context_tokens": 3500
|
||||
}
|
||||
self.filter._calculate_messages_tokens = lambda messages: len(messages) * 1000
|
||||
self.filter._count_tokens = lambda text: 1000
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "System prompt"},
|
||||
{"role": "user", "content": "Question 1"},
|
||||
{"role": "assistant", "content": "Answer 1"},
|
||||
{"role": "user", "content": "Question 2"},
|
||||
{"role": "assistant", "content": "Answer 2"},
|
||||
{"role": "user", "content": "Question 3"},
|
||||
]
|
||||
|
||||
asyncio.run(
|
||||
self.filter._generate_summary_async(
|
||||
messages=messages,
|
||||
chat_id="chat-1",
|
||||
body={"model": "fake-summary-model"},
|
||||
user_data={"id": "user-1"},
|
||||
target_compressed_count=5,
|
||||
lang="en-US",
|
||||
__event_emitter__=mock_emitter,
|
||||
__event_call__=None,
|
||||
)
|
||||
)
|
||||
|
||||
self.assertEqual(captured["chat_id"], "chat-1")
|
||||
self.assertEqual(captured["summary"], "new summary")
|
||||
self.assertEqual(captured["compressed_count"], 2)
|
||||
self.assertTrue(any(event["type"] == "status" for event in events))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,8 +1,10 @@
|
||||
[](https://openwebui.com/posts/async_context_compression_b1655bc8)
|
||||
|
||||
## Overview
|
||||
|
||||
**[🚀 Get/Update on OpenWebUI Community](https://openwebui.com/posts/async_context_compression_b1655bc8)**
|
||||
This release focuses on improving the structural integrity of chat history when using function-calling models and enhancing task reliability through concurrent task management. It introduces "Atomic Message Grouping" to prevent chat context corruption and a session-based locking mechanism to ensure stable background operations.
|
||||
|
||||
This release focuses on improving the structural integrity of chat history when using function-calling models and enhancing task reliability through concurrent task management. Version 1.4.0 introduces "Atomic Message Grouping" to prevent chat context corruption and a session-based locking mechanism to ensure stable background operations.
|
||||
**[📖 README](https://github.com/Fu-Jie/openwebui-extensions/blob/main/plugins/filters/async-context-compression/README.md)**
|
||||
|
||||
## New Features
|
||||
|
||||
@@ -20,7 +22,3 @@ This release focuses on improving the structural integrity of chat history when
|
||||
## Related Issues
|
||||
|
||||
- **[#56](https://github.com/Fu-Jie/openwebui-extensions/issues/56)**: Tool-Calling context corruption and concurrent summary tasks.
|
||||
|
||||
## Related PRs
|
||||
|
||||
- **[#61](https://github.com/Fu-Jie/openwebui-extensions/pull/61)**: (Placeholder) Full implementation of structure-aware grouping.
|
||||
|
||||
@@ -1,8 +1,6 @@
|
||||
## 概述
|
||||
[](https://openwebui.com/posts/async_context_compression_b1655bc8)
|
||||
|
||||
**[🚀 在 OpenWebUI 社区获取/更新](https://openwebui.com/posts/async_context_compression_b1655bc8)**
|
||||
|
||||
本次发布重点优化了在使用工具调用（Function Calling）模型时对话历史的结构完整性，并通过并发任务管理增强了系统的可靠性。1.4.0 版本引入了“原子消息组”逻辑以防止上下文损坏，并增加了会话级锁定机制以确保后台任务的稳定运行。
|
||||
本次发布重点优化了在使用工具调用（Function Calling）模型时对话历史的结构完整性，并通过并发任务管理增强了系统的可靠性。新版本引入了“原子消息组”逻辑以防止上下文损坏，并增加了会话级锁定机制以确保后台任务的稳定运行。
|
||||
|
||||
## 新功能
|
||||
|
||||
@@ -20,7 +18,3 @@
|
||||
## 相关 Issue
|
||||
|
||||
- **[#56](https://github.com/Fu-Jie/openwebui-extensions/issues/56)**: 修复工具调用上下文损坏及并发总结任务冲突问题。
|
||||
|
||||
## 相关 PR
|
||||
|
||||
- **[#61](https://github.com/Fu-Jie/openwebui-extensions/pull/61)**: (占位符) 结构感知消息分组的完整实现。
|
||||
|
||||
17
plugins/filters/async-context-compression/v1.4.1.md
Normal file
17
plugins/filters/async-context-compression/v1.4.1.md
Normal file
@@ -0,0 +1,17 @@
|
||||
[](https://openwebui.com/f/fujie/async_context_compression)
|
||||
|
||||
## Overview
|
||||
|
||||
This release addresses the critical progress coordinate drift issue in OpenWebUI's `outlet` phase, ensuring robust summarization for long tool-calling conversations.
|
||||
|
||||
[View on GitHub](https://github.com/Fu-Jie/openwebui-extensions/blob/main/plugins/filters/async-context-compression/README.md)
|
||||
|
||||
- **New Features**
|
||||
- **Reverse-Unfolding Mechanism**: Accurately reconstructs the expanded native tool-calling sequence during the outlet phase to permanently fix coordinate drift and missing summaries for long tool-based conversations.
|
||||
- **Safer Tool Trimming**: Refactored `enable_tool_output_trimming` to strictly use atomic block groups for safe trimming, completely preventing JSON payload corruption.
|
||||
|
||||
- **Bug Fixes**
|
||||
- Fixed coordinate drift where `compressed_message_count` could lose track due to OpenWebUI's frontend view truncating tool calls.
|
||||
|
||||
- **Related Issues**
|
||||
- Closes #56
|
||||
@@ -1,6 +1,6 @@
|
||||
# MkDocs Documentation Dependencies
|
||||
# Core MkDocs
|
||||
mkdocs>=1.5.0
|
||||
mkdocs>=1.5.0,<2.0.0
|
||||
|
||||
# Material Theme for MkDocs
|
||||
mkdocs-material>=9.5.0
|
||||
|
||||
26
scripts/.env.example
Normal file
26
scripts/.env.example
Normal file
@@ -0,0 +1,26 @@
|
||||
# OpenWebUI Bulk Installer Configuration
|
||||
#
|
||||
# Instructions:
|
||||
# - api_key: Copy from OpenWebUI Settings (starts with sk-)
|
||||
# - url: OpenWebUI server address (supports localhost, IP, and domain)
|
||||
#
|
||||
# URL Examples:
|
||||
# - Local: http://localhost:3000
|
||||
# - Remote IP: http://192.168.1.10:3000
|
||||
# - Domain: https://openwebui.example.com
|
||||
#
|
||||
# Environment variable precedence (highest to lowest):
|
||||
# 1. OPENWEBUI_API_KEY / OPENWEBUI_URL environment variables
|
||||
# 2. OPENWEBUI_BASE_URL environment variable
|
||||
# 3. Configuration in this .env file
|
||||
|
||||
# API Key (required)
|
||||
api_key=sk-your-api-key-here
|
||||
|
||||
# OpenWebUI server address (required)
|
||||
# Configure the baseURL where your OpenWebUI instance is running
|
||||
url=http://localhost:3000
|
||||
|
||||
# Alternatively, use environment variable format (both methods are equivalent)
|
||||
# OPENWEBUI_API_KEY=sk-your-api-key-here
|
||||
# OPENWEBUI_BASE_URL=http://localhost:3000
|
||||
@@ -1,147 +1,147 @@
|
||||
# 🚀 本地部署脚本指南 (Local Deployment Guide)
|
||||
# 🚀 Local Deployment Scripts Guide
|
||||
|
||||
## 概述
|
||||
## Overview
|
||||
|
||||
本目录包含用于将开发中的插件部署到本地 OpenWebUI 实例的自动化脚本。它们可以快速推送代码更改而无需重启 OpenWebUI。
|
||||
This directory contains automated scripts for deploying plugins in development to a local OpenWebUI instance. They enable quick code pushes without restarting OpenWebUI.
|
||||
|
||||
## 前置条件
|
||||
## Prerequisites
|
||||
|
||||
1. **OpenWebUI 运行中**: 确保 OpenWebUI 在本地运行（默认 `http://localhost:3003`）
|
||||
2. **API 密钥**: 需要一个有效的 OpenWebUI API 密钥
|
||||
3. **环境文件**: 在此目录创建 `.env` 文件，包含 API 密钥：
|
||||
1. **OpenWebUI Running**: Make sure OpenWebUI is running locally (default `http://localhost:3000`)
|
||||
2. **API Key**: You need a valid OpenWebUI API key
|
||||
3. **Environment File**: Create a `.env` file in this directory containing your API key:
|
||||
```
|
||||
api_key=sk-xxxxxxxxxxxxx
|
||||
```
|
||||
|
||||
## 快速开始
|
||||
## Quick Start
|
||||
|
||||
### 部署 Pipe 插件
|
||||
### Deploy a Pipe Plugin
|
||||
|
||||
```bash
|
||||
# 部署 GitHub Copilot SDK Pipe
|
||||
# Deploy GitHub Copilot SDK Pipe
|
||||
python deploy_pipe.py
|
||||
```
|
||||
|
||||
### 部署 Filter 插件
|
||||
### Deploy a Filter Plugin
|
||||
|
||||
```bash
|
||||
# 部署 async_context_compression Filter（默认）
|
||||
# Deploy async_context_compression Filter (default)
|
||||
python deploy_filter.py
|
||||
|
||||
# 部署指定的 Filter 插件
|
||||
# Deploy a specific Filter plugin
|
||||
python deploy_filter.py my-filter-name
|
||||
|
||||
# 列出所有可用的 Filter
|
||||
# List all available Filters
|
||||
python deploy_filter.py --list
|
||||
```
|
||||
|
||||
## 脚本说明
|
||||
## Script Documentation
|
||||
|
||||
### `deploy_filter.py` — Filter 插件部署工具
|
||||
### `deploy_filter.py` — Filter Plugin Deployment Tool
|
||||
|
||||
用于部署 Filter 类型的插件（如消息过滤、上下文压缩等）。
|
||||
Used to deploy Filter-type plugins (such as message filtering, context compression, etc.).
|
||||
|
||||
**主要特性**:
|
||||
- ✅ 从 Python 文件自动提取元数据（版本、作者、描述等）
|
||||
- ✅ 尝试更新现有插件，若不存在则创建新插件
|
||||
- ✅ 支持多个 Filter 插件管理
|
||||
- ✅ 详细的错误提示和连接诊断
|
||||
**Key Features**:
|
||||
- ✅ Auto-extracts metadata from Python files (version, author, description, etc.)
|
||||
- ✅ Attempts to update existing plugins, creates if not found
|
||||
- ✅ Supports multiple Filter plugin management
|
||||
- ✅ Detailed error messages and connection diagnostics
|
||||
|
||||
**用法**:
|
||||
**Usage**:
|
||||
```bash
|
||||
# 默认部署 async_context_compression
|
||||
# Deploy async_context_compression (default)
|
||||
python deploy_filter.py
|
||||
|
||||
# 部署其他 Filter
|
||||
# Deploy other Filters
|
||||
python deploy_filter.py async-context-compression
|
||||
python deploy_filter.py workflow-guide
|
||||
|
||||
# 列出所有可用 Filter
|
||||
# List all available Filters
|
||||
python deploy_filter.py --list
|
||||
python deploy_filter.py -l
|
||||
```
|
||||
|
||||
**工作流程**:
|
||||
1. 从 `.env` 加载 API 密钥
|
||||
2. 查找目标 Filter 插件目录
|
||||
3. 读取 Python 源文件
|
||||
4. 从 docstring 提取元数据（title, version, author, description, etc.）
|
||||
5. 构建 API 请求负载
|
||||
6. 发送更新请求到 OpenWebUI
|
||||
7. 若更新失败，自动尝试创建新插件
|
||||
8. 显示结果和诊断信息
|
||||
**Workflow**:
|
||||
1. Load API key from `.env`
|
||||
2. Find target Filter plugin directory
|
||||
3. Read Python source file
|
||||
4. Extract metadata from docstring (title, version, author, description, etc.)
|
||||
5. Build API request payload
|
||||
6. Send update request to OpenWebUI
|
||||
7. If update fails, auto-attempt to create new plugin
|
||||
8. Display results and diagnostic info
|
||||
|
||||
### `deploy_pipe.py` — Pipe 插件部署工具
|
||||
### `deploy_pipe.py` — Pipe Plugin Deployment Tool
|
||||
|
||||
用于部署 Pipe 类型的插件（如 GitHub Copilot SDK）。
|
||||
Used to deploy Pipe-type plugins (such as GitHub Copilot SDK).
|
||||
|
||||
**使用**:
|
||||
**Usage**:
|
||||
```bash
|
||||
python deploy_pipe.py
|
||||
```
|
||||
|
||||
## 获取 API 密钥
|
||||
## Get an API Key
|
||||
|
||||
### 方法 1: 使用现有用户令牌（推荐）
|
||||
### Method 1: Use Existing User Token (Recommended)
|
||||
|
||||
1. 打开 OpenWebUI 界面
|
||||
2. 点击用户头像 → Settings（设置）
|
||||
3. 找到 API Keys 部分
|
||||
4. 复制你的 API 密钥（sk-开头）
|
||||
5. 粘贴到 `.env` 文件中
|
||||
1. Open OpenWebUI interface
|
||||
2. Click user avatar → Settings
|
||||
3. Find the API Keys section
|
||||
4. Copy your API key (starts with sk-)
|
||||
5. Paste into `.env` file
|
||||
|
||||
### 方法 2: 创建长期 API 密钥
|
||||
### Method 2: Create a Long-term API Key
|
||||
|
||||
在 OpenWebUI 设置中创建专用于部署的长期 API 密钥。
|
||||
Create a dedicated long-term API key in OpenWebUI Settings for deployment purposes.
|
||||
|
||||
## 故障排除
|
||||
## Troubleshooting
|
||||
|
||||
### "Connection error: Could not reach OpenWebUI at localhost:3003"
|
||||
### "Connection error: Could not reach OpenWebUI at localhost:3000"
|
||||
|
||||
**原因**: OpenWebUI 未运行或端口不同
|
||||
**Cause**: OpenWebUI is not running or port is different
|
||||
|
||||
**解决方案**:
|
||||
- 确保 OpenWebUI 正在运行
|
||||
- 检查 OpenWebUI 实际监听的端口（通常是 3000 或 3003）
|
||||
- 根据需要编辑脚本中的 URL
|
||||
**Solution**:
|
||||
- Make sure OpenWebUI is running
|
||||
- Check which port OpenWebUI is actually listening on (usually 3000)
|
||||
- Edit the URL in the script if needed
|
||||
|
||||
### ".env file not found"
|
||||
|
||||
**原因**: 未创建 `.env` 文件
|
||||
**Cause**: `.env` file was not created
|
||||
|
||||
**解决方案**:
|
||||
**Solution**:
|
||||
```bash
|
||||
echo "api_key=sk-your-api-key-here" > .env
|
||||
```
|
||||
|
||||
### "Filter 'xxx' not found"
|
||||
|
||||
**原因**: Filter 目录名不正确
|
||||
**Cause**: Filter directory name is incorrect
|
||||
|
||||
**解决方案**:
|
||||
**Solution**:
|
||||
```bash
|
||||
# 列出所有可用的 Filter
|
||||
# List all available Filters
|
||||
python deploy_filter.py --list
|
||||
```
|
||||
|
||||
### "Failed to update or create. Status: 401"
|
||||
|
||||
**原因**: API 密钥无效或过期
|
||||
**Cause**: API key is invalid or expired
|
||||
|
||||
**解决方案**:
|
||||
1. 验证 API 密钥的有效性
|
||||
2. 获取新的 API 密钥
|
||||
3. 更新 `.env` 文件
|
||||
**Solution**:
|
||||
1. Verify your API key is valid
|
||||
2. Generate a new API key
|
||||
3. Update the `.env` file
|
||||
|
||||
## 工作流示例
|
||||
## Workflow Examples
|
||||
|
||||
### 开发并部署新的 Filter
|
||||
### Develop and Deploy a New Filter
|
||||
|
||||
```bash
|
||||
# 1. 在 plugins/filters/ 创建新的 Filter 目录
|
||||
# 1. Create new Filter directory in plugins/filters/
|
||||
mkdir plugins/filters/my-new-filter
|
||||
|
||||
# 2. 创建 my_new_filter.py 文件，包含必要的元数据：
|
||||
# 2. Create my_new_filter.py with required metadata:
|
||||
# """
|
||||
# title: My New Filter
|
||||
# author: Your Name
|
||||
@@ -149,58 +149,58 @@ mkdir plugins/filters/my-new-filter
|
||||
# description: Filter description
|
||||
# """
|
||||
|
||||
# 3. 部署到本地 OpenWebUI
|
||||
# 3. Deploy to local OpenWebUI
|
||||
cd scripts
|
||||
python deploy_filter.py my-new-filter
|
||||
|
||||
# 4. 在 OpenWebUI UI 中测试插件
|
||||
# 4. Test the plugin in OpenWebUI UI
|
||||
|
||||
# 5. 继续迭代开发
|
||||
# ... 修改代码 ...
|
||||
# 5. Continue development
|
||||
# ... modify code ...
|
||||
|
||||
# 6. 重新部署（自动覆盖）
|
||||
# 6. Re-deploy (auto-overwrites)
|
||||
python deploy_filter.py my-new-filter
|
||||
```
|
||||
|
||||
### 修复 Bug 并快速部署
|
||||
### Fix a Bug and Deploy Quickly
|
||||
|
||||
```bash
|
||||
# 1. 修改源代码
|
||||
# 1. Modify the source code
|
||||
# vim ../plugins/filters/async-context-compression/async_context_compression.py
|
||||
|
||||
# 2. 立即部署到本地
|
||||
# 2. Deploy immediately to local
|
||||
python deploy_filter.py async-context-compression
|
||||
|
||||
# 3. 在 OpenWebUI 中测试修复
|
||||
# （无需重启 OpenWebUI）
|
||||
# 3. Test the fix in OpenWebUI
|
||||
# (No need to restart OpenWebUI)
|
||||
```
|
||||
|
||||
## 安全注意事项
|
||||
## Security Considerations
|
||||
|
||||
⚠️ **重要**:
|
||||
- ✅ 将 `.env` 文件添加到 `.gitignore`（避免提交敏感信息）
|
||||
- ✅ 不要在版本控制中提交 API 密钥
|
||||
- ✅ 仅在可信的网络环境中使用
|
||||
- ✅ 定期轮换 API 密钥
|
||||
⚠️ **Important**:
|
||||
- ✅ Add `.env` file to `.gitignore` (avoid committing sensitive info)
|
||||
- ✅ Never commit API keys to version control
|
||||
- ✅ Use only on trusted networks
|
||||
- ✅ Rotate API keys periodically
|
||||
|
||||
## 文件结构
|
||||
## File Structure
|
||||
|
||||
```
|
||||
scripts/
|
||||
├── deploy_filter.py # Filter 插件部署工具
|
||||
├── deploy_pipe.py # Pipe 插件部署工具
|
||||
├── .env # API 密钥（本地，不提交）
|
||||
├── README.md # 本文件
|
||||
├── deploy_filter.py # Filter plugin deployment tool
|
||||
├── deploy_pipe.py # Pipe plugin deployment tool
|
||||
├── .env # API key (local, not committed)
|
||||
├── README.md # This file
|
||||
└── ...
|
||||
```
|
||||
|
||||
## 参考资源
|
||||
## Reference Resources
|
||||
|
||||
- [OpenWebUI 文档](https://docs.openwebui.com/)
|
||||
- [插件开发指南](../docs/development/plugin-guide.md)
|
||||
- [Filter 插件示例](../plugins/filters/)
|
||||
- [OpenWebUI Documentation](https://docs.openwebui.com/)
|
||||
- [Plugin Development Guide](../docs/development/plugin-guide.md)
|
||||
- [Filter Plugin Examples](../plugins/filters/)
|
||||
|
||||
---
|
||||
|
||||
**最后更新**: 2026-03-09
|
||||
**作者**: Fu-Jie
|
||||
**Last Updated**: 2026-03-09
|
||||
**Author**: Fu-Jie
|
||||
|
||||
@@ -1,114 +1,114 @@
|
||||
# 📦 Async Context Compression — 本地部署工具 (Local Deployment Tools)
|
||||
# 📦 Async Context Compression — Local Deployment Tools
|
||||
|
||||
## 🎯 功能概述
|
||||
## 🎯 Feature Overview
|
||||
|
||||
为 `async_context_compression` Filter 插件添加了完整的本地部署工具链，支持快速迭代开发无需重启 OpenWebUI。
|
||||
Added a complete local deployment toolchain for the `async_context_compression` Filter plugin, supporting fast iterative development without restarting OpenWebUI.
|
||||
|
||||
## 📋 新增文件
|
||||
## 📋 New Files
|
||||
|
||||
### 1. **deploy_filter.py** — Filter 插件部署脚本
|
||||
- **位置**: `scripts/deploy_filter.py`
|
||||
- **功能**: 自动部署 Filter 类插件到本地 OpenWebUI 实例
|
||||
- **特性**:
|
||||
- ✅ 从 Python docstring 自动提取元数据
|
||||
- ✅ 智能版本号识别（semantic versioning）
|
||||
- ✅ 支持多个 Filter 插件管理
|
||||
- ✅ 自动更新或创建插件
|
||||
- ✅ 详细的错误诊断和连接测试
|
||||
- ✅ 列表指令查看所有可用 Filter
|
||||
- **代码行数**: ~300 行
|
||||
### 1. **deploy_filter.py** — Filter Plugin Deployment Script
|
||||
- **Location**: `scripts/deploy_filter.py`
|
||||
- **Function**: Auto-deploy Filter-type plugins to local OpenWebUI instance
|
||||
- **Features**:
|
||||
- ✅ Auto-extract metadata from Python docstring
|
||||
- ✅ Smart semantic version recognition
|
||||
- ✅ Support multiple Filter plugin management
|
||||
- ✅ Auto-update or create plugins
|
||||
- ✅ Detailed error diagnostics and connection testing
|
||||
- ✅ List command to view all available Filters
|
||||
- **Code Lines**: ~300
|
||||
|
||||
### 2. **DEPLOYMENT_GUIDE.md** — 完整部署指南
|
||||
- **位置**: `scripts/DEPLOYMENT_GUIDE.md`
|
||||
- **内容**:
|
||||
- 前置条件和快速开始
|
||||
- 脚本详细说明
|
||||
- API 密钥获取方法
|
||||
- 故障排除指南
|
||||
- 分步工作流示例
|
||||
### 2. **DEPLOYMENT_GUIDE.md** — Complete Deployment Guide
|
||||
- **Location**: `scripts/DEPLOYMENT_GUIDE.md`
|
||||
- **Contents**:
|
||||
- Prerequisites and quick start
|
||||
- Detailed script documentation
|
||||
- API key retrieval method
|
||||
- Troubleshooting guide
|
||||
- Step-by-step workflow examples
|
||||
|
||||
### 3. **QUICK_START.md** — 快速参考卡片
|
||||
- **位置**: `scripts/QUICK_START.md`
|
||||
- **内容**:
|
||||
- 一行命令部署
|
||||
- 前置步骤
|
||||
- 常见命令表格
|
||||
- 故障诊断速查表
|
||||
- CI/CD 集成示例
|
||||
### 3. **QUICK_START.md** — Quick Reference Card
|
||||
- **Location**: `scripts/QUICK_START.md`
|
||||
- **Contents**:
|
||||
- One-line deployment command
|
||||
- Setup steps
|
||||
- Common commands table
|
||||
- Troubleshooting quick-reference table
|
||||
- CI/CD integration examples
|
||||
|
||||
### 4. **test_deploy_filter.py** — 单元测试套件
|
||||
- **位置**: `tests/scripts/test_deploy_filter.py`
|
||||
- **测试覆盖**:
|
||||
- ✅ Filter 文件发现 (3 个测试)
|
||||
- ✅ 元数据提取 (3 个测试)
|
||||
- ✅ API 负载构建 (4 个测试)
|
||||
- **测试通过率**: 10/10 ✅
|
||||
### 4. **test_deploy_filter.py** — Unit Test Suite
|
||||
- **Location**: `tests/scripts/test_deploy_filter.py`
|
||||
- **Test Coverage**:
|
||||
- ✅ Filter file discovery (3 tests)
|
||||
- ✅ Metadata extraction (3 tests)
|
||||
- ✅ API payload building (4 tests)
|
||||
- **Pass Rate**: 10/10 ✅
|
||||
|
||||
## 🚀 使用方式
|
||||
## 🚀 Usage
|
||||
|
||||
### 基本部署（一行命令）
|
||||
### Basic Deploy (One-liner)
|
||||
|
||||
```bash
|
||||
cd scripts
|
||||
python deploy_filter.py
|
||||
```
|
||||
|
||||
### 列出所有可用 Filter
|
||||
### List All Available Filters
|
||||
|
||||
```bash
|
||||
python deploy_filter.py --list
|
||||
```
|
||||
|
||||
### 部署指定 Filter
|
||||
### Deploy Specific Filter
|
||||
|
||||
```bash
|
||||
python deploy_filter.py folder-memory
|
||||
python deploy_filter.py context_enhancement_filter
|
||||
```
|
||||
|
||||
## 🔧 工作原理
|
||||
## 🔧 How It Works
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 1. 加载 API 密钥 (.env) │
|
||||
│ 1. Load API key (.env) │
|
||||
└──────────────────┬──────────────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────▼──────────────────────────────────────────┐
|
||||
│ 2. 查找 Filter 插件文件 │
|
||||
│ - 从名称推断文件路径 │
|
||||
│ - 支持 hyphen-case 和 snake_case 查找 │
|
||||
│ 2. Find Filter plugin file │
|
||||
│ - Infer file path from name │
|
||||
│ - Support hyphen-case and snake_case lookup │
|
||||
└──────────────────┬──────────────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────▼──────────────────────────────────────────┐
|
||||
│ 3. 读取 Python 源代码 │
|
||||
│ - 提取 docstring 元数据 │
|
||||
│ 3. Read Python source code │
|
||||
│ - Extract docstring metadata │
|
||||
│ - title, version, author, description, openwebui_id │
|
||||
└──────────────────┬──────────────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────▼──────────────────────────────────────────┐
|
||||
│ 4. 构建 API 请求负载 │
|
||||
│ - 组装 manifest 和 meta 信息 │
|
||||
│ - 包含完整源代码内容 │
|
||||
│ 4. Build API request payload │
|
||||
│ - Assemble manifest and meta info │
|
||||
│ - Include complete source code content │
|
||||
└──────────────────┬──────────────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────▼──────────────────────────────────────────┐
|
||||
│ 5. 发送请求 │
|
||||
│ - POST /api/v1/functions/id/{id}/update （更新） │
|
||||
│ - POST /api/v1/functions/create （创建备用） │
|
||||
│ 5. Send request │
|
||||
│ - POST /api/v1/functions/id/{id}/update (update) │
|
||||
│ - POST /api/v1/functions/create (create fallback) │
|
||||
└──────────────────┬──────────────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────▼──────────────────────────────────────────┐
|
||||
│ 6. 显示结果和诊断 │
|
||||
│ - ✅ 更新/创建成功 │
|
||||
│ - ❌ 错误信息和解决建议 │
|
||||
│ 6. Display results and diagnostics │
|
||||
│ - ✅ Update/create success │
|
||||
│ - ❌ Error messages and solutions │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 📊 支持的 Filter 列表
|
||||
## 📊 Supported Filters List
|
||||
|
||||
脚本自动发现以下 Filter：
|
||||
Script auto-discovers the following Filters:
|
||||
|
||||
| Filter 名称 | Python 文件 | 版本 |
|
||||
| Filter Name | Python File | Version |
|
||||
|-----------|-----------|------|
|
||||
| async-context-compression | async_context_compression.py | 1.3.0+ |
|
||||
| chat-session-mapping-filter | chat_session_mapping_filter.py | 0.1.0+ |
|
||||
@@ -118,11 +118,11 @@ python deploy_filter.py context_enhancement_filter
|
||||
| markdown_normalizer | markdown_normalizer.py | 1.2.8+ |
|
||||
| web_gemini_multimodel_filter | web_gemini_multimodel_filter.py | 0.3.2+ |
|
||||
|
||||
## ⚙️ 技术细节
|
||||
## ⚙️ Technical Details
|
||||
|
||||
### 元数据提取
|
||||
### Metadata Extraction
|
||||
|
||||
脚本从 Python 文件顶部的 docstring 中提取元数据：
|
||||
Script extracts metadata from the docstring at the top of Python file:
|
||||
|
||||
```python
|
||||
"""
|
||||
@@ -137,111 +137,111 @@ openwebui_id: b1655bc8-6de9-4cad-8cb5-a6f7829a02ce
|
||||
"""
|
||||
```
|
||||
|
||||
**支持的元数据字段**:
|
||||
- `title` — Filter 显示名称 ✅
|
||||
- `id` — 唯一标识符 ✅
|
||||
- `author` — 作者名称 ✅
|
||||
- `author_url` — 作者主页链接 ✅
|
||||
- `funding_url` — 项目链接 ✅
|
||||
- `description` — 功能描述 ✅
|
||||
- `version` — 语义化版本号 ✅
|
||||
- `openwebui_id` — OpenWebUI UUID （可选）
|
||||
**Supported Metadata Fields**:
|
||||
- `title` — Filter display name ✅
|
||||
- `id` — Unique identifier ✅
|
||||
- `author` — Author name ✅
|
||||
- `author_url` — Author homepage ✅
|
||||
- `funding_url` — Project link ✅
|
||||
- `description` — Feature description ✅
|
||||
- `version` — Semantic version number ✅
|
||||
- `openwebui_id` — OpenWebUI UUID (optional)
|
||||
|
||||
### API 集成
|
||||
### API Integration
|
||||
|
||||
脚本使用 OpenWebUI REST API：
|
||||
Script uses OpenWebUI REST API:
|
||||
|
||||
```
|
||||
POST /api/v1/functions/id/{filter_id}/update
|
||||
- 更新现有 Filter
|
||||
- HTTP 200: 更新成功
|
||||
- HTTP 404: Filter 不存在，自动尝试创建
|
||||
- Update existing Filter
|
||||
- HTTP 200: Update success
|
||||
- HTTP 404: Filter not found, auto-attempt create
|
||||
|
||||
POST /api/v1/functions/create
|
||||
- 创建新 Filter
|
||||
- HTTP 200: 创建成功
|
||||
- Create new Filter
|
||||
- HTTP 200: Creation success
|
||||
```
|
||||
|
||||
**认证**: Bearer token (API 密钥方式)
|
||||
**Authentication**: Bearer token (API key method)
|
||||
|
||||
## 🔐 安全性
|
||||
## 🔐 Security
|
||||
|
||||
### API 密钥管理
|
||||
### API Key Management
|
||||
|
||||
```bash
|
||||
# 1. 创建 .env 文件
|
||||
# 1. Create .env file
|
||||
echo "api_key=sk-your-key-here" > scripts/.env
|
||||
|
||||
# 2. 将 .env 添加到 .gitignore
|
||||
# 2. Add .env to .gitignore
|
||||
echo "scripts/.env" >> .gitignore
|
||||
|
||||
# 3. 不要提交 API 密钥
|
||||
# 3. Don't commit API key
|
||||
git add scripts/.gitignore
|
||||
git commit -m "chore: add .env to gitignore"
|
||||
```
|
||||
|
||||
### 最佳实践
|
||||
### Best Practices
|
||||
|
||||
- ✅ 使用长期认证令牌（而不是短期 JWT）
|
||||
- ✅ 定期轮换 API 密钥
|
||||
- ✅ 限制密钥权限范围
|
||||
- ✅ 在可信网络中使用
|
||||
- ✅ 生产环境使用 CI/CD 秘密管理
|
||||
- ✅ Use long-term auth tokens (not short-term JWT)
|
||||
- ✅ Rotate API keys periodically
|
||||
- ✅ Limit key permission scope
|
||||
- ✅ Use only on trusted networks
|
||||
- ✅ Use CI/CD secret management in production
|
||||
|
||||
## 🧪 测试验证
|
||||
## 🧪 Test Verification
|
||||
|
||||
### 运行测试套件
|
||||
### Run Test Suite
|
||||
|
||||
```bash
|
||||
pytest tests/scripts/test_deploy_filter.py -v
|
||||
```
|
||||
|
||||
### 测试覆盖范围
|
||||
### Test Coverage
|
||||
|
||||
```
|
||||
✅ TestFilterDiscovery (3 个测试)
|
||||
✅ TestFilterDiscovery (3 tests)
|
||||
- test_find_async_context_compression
|
||||
- test_find_nonexistent_filter
|
||||
- test_find_filter_with_underscores
|
||||
|
||||
✅ TestMetadataExtraction (3 个测试)
|
||||
✅ TestMetadataExtraction (3 tests)
|
||||
- test_extract_metadata_from_async_compression
|
||||
- test_extract_metadata_empty_file
|
||||
- test_extract_metadata_multiline_docstring
|
||||
|
||||
✅ TestPayloadBuilding (4 个测试)
|
||||
✅ TestPayloadBuilding (4 tests)
|
||||
- test_build_filter_payload_basic
|
||||
- test_payload_has_required_fields
|
||||
- test_payload_with_openwebui_id
|
||||
|
||||
✅ TestVersionExtraction (1 个测试)
|
||||
✅ TestVersionExtraction (1 test)
|
||||
- test_extract_valid_version
|
||||
|
||||
结果: 10/10 通过 ✅
|
||||
Result: 10/10 PASSED ✅
|
||||
```
|
||||
|
||||
## 💡 常见用例
|
||||
## 💡 Common Use Cases
|
||||
|
||||
### 用例 1: 修复 Bug 后快速测试
|
||||
### Use Case 1: Quick Test After Bug Fix
|
||||
|
||||
```bash
|
||||
# 1. 修改代码
|
||||
# 1. Modify code
|
||||
vim plugins/filters/async-context-compression/async_context_compression.py
|
||||
|
||||
# 2. 立即部署（不需要重启 OpenWebUI）
|
||||
# 2. Deploy immediately (no OpenWebUI restart needed)
|
||||
cd scripts && python deploy_filter.py
|
||||
|
||||
# 3. 在 OpenWebUI 中测试修复
|
||||
# 4. 重复迭代（返回步骤 1）
|
||||
# 3. Test fix in OpenWebUI
|
||||
# 4. Iterate (return to step 1)
|
||||
```
|
||||
|
||||
### 用例 2: 开发新的 Filter
|
||||
### Use Case 2: Develop New Filter
|
||||
|
||||
```bash
|
||||
# 1. 创建新 Filter 目录
|
||||
# 1. Create new Filter directory
|
||||
mkdir plugins/filters/my-new-filter
|
||||
|
||||
# 2. 编写代码（包含必要的 docstring 元数据）
|
||||
# 2. Write code (include required docstring metadata)
|
||||
cat > plugins/filters/my-new-filter/my_new_filter.py << 'EOF'
|
||||
"""
|
||||
title: My New Filter
|
||||
@@ -254,33 +254,33 @@ class Filter:
|
||||
# ... implementation ...
|
||||
EOF
|
||||
|
||||
# 3. 首次部署（创建）
|
||||
# 3. First deployment (create)
|
||||
cd scripts && python deploy_filter.py my-new-filter
|
||||
|
||||
# 4. 在 OpenWebUI UI 测试
|
||||
# 5. 重复更新
|
||||
# 4. Test in OpenWebUI UI
|
||||
# 5. Repeat updates
|
||||
cd scripts && python deploy_filter.py my-new-filter
|
||||
```
|
||||
|
||||
### 用例 3: 版本更新和发布
|
||||
### Use Case 3: Version Update and Release
|
||||
|
||||
```bash
|
||||
# 1. 更新版本号
|
||||
# 1. Update version number
|
||||
vim plugins/filters/async-context-compression/async_context_compression.py
|
||||
# 修改: version: 1.3.0 → version: 1.4.0
|
||||
# Change: version: 1.3.0 → version: 1.4.0
|
||||
|
||||
# 2. 部署新版本
|
||||
# 2. Deploy new version
|
||||
cd scripts && python deploy_filter.py
|
||||
|
||||
# 3. 测试通过后提交
|
||||
# 3. After testing, commit
|
||||
git add plugins/filters/async-context-compression/
|
||||
git commit -m "feat(filters): update async-context-compression to 1.4.0"
|
||||
git push
|
||||
```
|
||||
|
||||
## 🔄 CI/CD 集成
|
||||
## 🔄 CI/CD Integration
|
||||
|
||||
### GitHub Actions 示例
|
||||
### GitHub Actions Example
|
||||
|
||||
```yaml
|
||||
name: Deploy Filter on Release
|
||||
@@ -308,71 +308,71 @@ jobs:
|
||||
api_key: ${{ secrets.OPENWEBUI_API_KEY }}
|
||||
```
|
||||
|
||||
## 📚 参考文档
|
||||
## 📚 Reference Documentation
|
||||
|
||||
- [完整部署指南](DEPLOYMENT_GUIDE.md)
|
||||
- [快速参考卡片](QUICK_START.md)
|
||||
- [测试套件](../tests/scripts/test_deploy_filter.py)
|
||||
- [插件开发指南](../docs/development/plugin-guide.md)
|
||||
- [OpenWebUI 文档](https://docs.openwebui.com/)
|
||||
- [Complete Deployment Guide](DEPLOYMENT_GUIDE.md)
|
||||
- [Quick Reference Card](QUICK_START.md)
|
||||
- [Test Suite](../tests/scripts/test_deploy_filter.py)
|
||||
- [Plugin Development Guide](../docs/development/plugin-guide.md)
|
||||
- [OpenWebUI Documentation](https://docs.openwebui.com/)
|
||||
|
||||
## 🎓 学习资源
|
||||
## 🎓 Learning Resources
|
||||
|
||||
### 架构理解
|
||||
### Architecture Understanding
|
||||
|
||||
```
|
||||
OpenWebUI 系统设计
|
||||
OpenWebUI System Design
|
||||
↓
|
||||
Filter 插件类型定义
|
||||
Filter Plugin Type Definition
|
||||
↓
|
||||
REST API 接口 (/api/v1/functions)
|
||||
REST API Interface (/api/v1/functions)
|
||||
↓
|
||||
本地部署脚本实现 (deploy_filter.py)
|
||||
Local Deployment Script Implementation (deploy_filter.py)
|
||||
↓
|
||||
元数据提取和投递
|
||||
Metadata Extraction and Delivery
|
||||
```
|
||||
|
||||
### 调试技巧
|
||||
### Debugging Tips
|
||||
|
||||
1. **启用详细日志**:
|
||||
1. **Enable Verbose Logging**:
|
||||
```bash
|
||||
python deploy_filter.py 2>&1 | tee deploy.log
|
||||
```
|
||||
|
||||
2. **测试 API 连接**:
|
||||
2. **Test API Connection**:
|
||||
```bash
|
||||
curl -X GET http://localhost:3003/api/v1/functions \
|
||||
curl -X GET http://localhost:3000/api/v1/functions \
|
||||
-H "Authorization: Bearer $API_KEY"
|
||||
```
|
||||
|
||||
3. **验证 .env 文件**:
|
||||
3. **Verify .env File**:
|
||||
```bash
|
||||
grep "api_key=" scripts/.env
|
||||
```
|
||||
|
||||
## 📞 故障排除
|
||||
## 📞 Troubleshooting
|
||||
|
||||
| 问题 | 诊断 | 解决方案 |
|
||||
|------|------|----------|
|
||||
| Connection error | OpenWebUI 地址/端口不对 | 检查 localhost:3003；修改 URL 如需要 |
|
||||
| .env not found | 未创建配置文件 | `echo "api_key=sk-..." > scripts/.env` |
|
||||
| Filter not found | 插件名称错误 | 运行 `python deploy_filter.py --list` |
|
||||
| Status 401 | API 密钥无效/过期 | 更新 `.env` 中的密钥 |
|
||||
| Status 500 | 服务器错误 | 检查 OpenWebUI 服务日志 |
|
||||
| Issue | Diagnosis | Solution |
|
||||
|-------|-----------|----------|
|
||||
| Connection error | Wrong OpenWebUI address/port | Check localhost:3000; modify URL if needed |
|
||||
| .env not found | Config file not created | `echo "api_key=sk-..." > scripts/.env` |
|
||||
| Filter not found | Wrong Plugin name | Run `python deploy_filter.py --list` |
|
||||
| Status 401 | Invalid/expired API key | Update key in `.env` |
|
||||
| Status 500 | Server error | Check OpenWebUI service logs |
|
||||
|
||||
## ✨ 特色功能
|
||||
## ✨ Highlight Features
|
||||
|
||||
| 特性 | 描述 |
|
||||
|------|------|
|
||||
| 🔍 自动发现 | 自动查找所有 Filter 插件 |
|
||||
| 📊 元数据提取 | 从代码自动提取版本和元数据 |
|
||||
| ♻️ 自动更新 | 智能处理更新或创建 |
|
||||
| 🛡️ 错误处理 | 详细的错误提示和诊断信息 |
|
||||
| 🚀 快速迭代 | 秒级部署，无需重启 |
|
||||
| 🧪 完整测试 | 10 个单元测试覆盖核心功能 |
|
||||
| Feature | Description |
|
||||
|---------|-------------|
|
||||
| 🔍 Auto Discovery | Automatically find all Filter plugins |
|
||||
| 📊 Metadata Extraction | Auto-extract version and metadata from code |
|
||||
| ♻️ Auto-update | Smart handling of update or create |
|
||||
| 🛡️ Error Handling | Detailed error messages and diagnostics |
|
||||
| 🚀 Fast Iteration | Second-level deployment, no restart |
|
||||
| 🧪 Complete Testing | 10 unit tests covering core functions |
|
||||
|
||||
---
|
||||
|
||||
**最后更新**: 2026-03-09
|
||||
**作者**: Fu-Jie
|
||||
**项目**: [openwebui-extensions](https://github.com/Fu-Jie/openwebui-extensions)
|
||||
**Last Updated**: 2026-03-09
|
||||
**Author**: Fu-Jie
|
||||
**Project**: [openwebui-extensions](https://github.com/Fu-Jie/openwebui-extensions)
|
||||
|
||||
@@ -1,76 +1,76 @@
|
||||
# ⚡ 快速部署参考 (Quick Deployment Reference)
|
||||
# ⚡ Quick Deployment Reference
|
||||
|
||||
## 一行命令部署
|
||||
## One-line Deploy Commands
|
||||
|
||||
```bash
|
||||
# 部署 async_context_compression Filter（默认）
|
||||
# Deploy async_context_compression Filter (default)
|
||||
cd scripts && python deploy_filter.py
|
||||
|
||||
# 列出所有可用 Filter
|
||||
# List all available Filters
|
||||
cd scripts && python deploy_filter.py --list
|
||||
```
|
||||
|
||||
## 前置步骤（仅需一次）
|
||||
## Setup Steps (One time only)
|
||||
|
||||
```bash
|
||||
# 1. 进入 scripts 目录
|
||||
# 1. Enter scripts directory
|
||||
cd scripts
|
||||
|
||||
# 2. 创建 .env 文件，包含 OpenWebUI API 密钥
|
||||
# 2. Create .env file with your OpenWebUI API key
|
||||
echo "api_key=sk-your-api-key-here" > .env
|
||||
|
||||
# 3. 确保 OpenWebUI 运行在 localhost:3003
|
||||
# 3. Make sure OpenWebUI is running on localhost:3000
|
||||
```
|
||||
|
||||
## 获取 API 密钥
|
||||
## Get Your API Key
|
||||
|
||||
1. 打开 OpenWebUI → 用户头像 → Settings
|
||||
2. 找到 "API Keys" 部分
|
||||
3. 复制密钥（sk-开头）
|
||||
4. 粘贴到 `.env` 文件
|
||||
1. Open OpenWebUI → user avatar → Settings
|
||||
2. Find "API Keys" section
|
||||
3. Copy your key (starts with sk-)
|
||||
4. Paste into `.env` file
|
||||
|
||||
## 部署流程
|
||||
## Deployment Workflow
|
||||
|
||||
```bash
|
||||
# 1. 编辑插件代码
|
||||
# 1. Edit plugin code
|
||||
vim ../plugins/filters/async-context-compression/async_context_compression.py
|
||||
|
||||
# 2. 部署到本地
|
||||
# 2. Deploy to local
|
||||
python deploy_filter.py
|
||||
|
||||
# 3. 在 OpenWebUI 测试（无需重启）
|
||||
# 3. Test in OpenWebUI (no restart needed)
|
||||
|
||||
# 4. 重复部署（自动覆盖）
|
||||
# 4. Deploy again (auto-overwrites)
|
||||
python deploy_filter.py
|
||||
```
|
||||
|
||||
## 常见命令
|
||||
## Common Commands
|
||||
|
||||
| 命令 | 说明 |
|
||||
|------|------|
|
||||
| `python deploy_filter.py` | 部署 async_context_compression |
|
||||
| `python deploy_filter.py filter-name` | 部署指定 Filter |
|
||||
| `python deploy_filter.py --list` | 列出所有可用 Filter |
|
||||
| `python deploy_pipe.py` | 部署 GitHub Copilot SDK Pipe |
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `python deploy_filter.py` | Deploy async_context_compression |
|
||||
| `python deploy_filter.py filter-name` | Deploy specific Filter |
|
||||
| `python deploy_filter.py --list` | List all available Filters |
|
||||
| `python deploy_pipe.py` | Deploy GitHub Copilot SDK Pipe |
|
||||
|
||||
## 故障诊断
|
||||
## Troubleshooting
|
||||
|
||||
| 错误 | 原因 | 解决方案 |
|
||||
|------|------|----------|
|
||||
| Connection error | OpenWebUI 未运行 | 启动 OpenWebUI 或检查端口 |
|
||||
| .env not found | 未创建配置文件 | `echo "api_key=sk-..." > .env` |
|
||||
| Filter not found | Filter 名称错误 | 运行 `python deploy_filter.py --list` |
|
||||
| Status 401 | API 密钥无效 | 更新 `.env` 中的密钥 |
|
||||
| Error | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| Connection error | OpenWebUI not running | Start OpenWebUI or check port |
|
||||
| .env not found | Config file not created | `echo "api_key=sk-..." > .env` |
|
||||
| Filter not found | Filter name is wrong | Run `python deploy_filter.py --list` |
|
||||
| Status 401 | API key invalid | Update key in `.env` |
|
||||
|
||||
## 文件位置
|
||||
## File Locations
|
||||
|
||||
```
|
||||
openwebui-extensions/
|
||||
├── scripts/
|
||||
│ ├── deploy_filter.py ← Filter 部署工具
|
||||
│ ├── deploy_pipe.py ← Pipe 部署工具
|
||||
│ ├── .env ← API 密钥（不提交）
|
||||
│ └── DEPLOYMENT_GUIDE.md ← 完整指南
|
||||
│ ├── deploy_filter.py ← Filter deployment tool
|
||||
│ ├── deploy_pipe.py ← Pipe deployment tool
|
||||
│ ├── .env ← API key (don't commit)
|
||||
│ └── DEPLOYMENT_GUIDE.md ← Full guide
|
||||
│
|
||||
└── plugins/
|
||||
└── filters/
|
||||
@@ -80,26 +80,26 @@ openwebui-extensions/
|
||||
└── README_CN.md
|
||||
```
|
||||
|
||||
## 工作流建议
|
||||
## Suggested Workflow
|
||||
|
||||
### 快速迭代开发
|
||||
### Fast Iterative Development
|
||||
|
||||
```bash
|
||||
# Terminal 1: 启动 OpenWebUI（如果未运行）
|
||||
docker run -d -p 3003:8080 ghcr.io/open-webui/open-webui:latest
|
||||
# Terminal 1: Start OpenWebUI (if not running)
|
||||
docker run -d -p 3000:8080 ghcr.io/open-webui/open-webui:latest
|
||||
|
||||
# Terminal 2: 开发环节（重复执行）
|
||||
# Terminal 2: Development loop (repeated)
|
||||
cd scripts
|
||||
code ../plugins/filters/async-context-compression/ # 编辑代码
|
||||
python deploy_filter.py # 部署
|
||||
# → 在 OpenWebUI 测试
|
||||
# → 返回编辑，重复
|
||||
code ../plugins/filters/async-context-compression/ # Edit code
|
||||
python deploy_filter.py # Deploy
|
||||
# → Test in OpenWebUI
|
||||
# → Go back to edit, repeat
|
||||
```
|
||||
|
||||
### CI/CD 集成
|
||||
### CI/CD Integration
|
||||
|
||||
```bash
|
||||
# 在 GitHub Actions 中
|
||||
# In GitHub Actions
|
||||
- name: Deploy filter to staging
|
||||
run: |
|
||||
cd scripts
|
||||
@@ -110,4 +110,4 @@ python deploy_filter.py # 部署
|
||||
|
||||
---
|
||||
|
||||
📚 **更多帮助**: 查看 `DEPLOYMENT_GUIDE.md`
|
||||
📚 **More Help**: See `DEPLOYMENT_GUIDE.md`
|
||||
|
||||
@@ -1,70 +1,84 @@
|
||||
# 🚀 部署脚本使用指南 (Deployment Scripts Guide)
|
||||
# 🚀 Deployment Scripts Guide
|
||||
|
||||
## 📁 新增部署工具
|
||||
## 📁 Deployment Tools
|
||||
|
||||
为了支持快速本地部署 async_context_compression 和其他 Filter 插件，我们添加了以下文件：
|
||||
To support quick local deployment of async_context_compression and other Filter plugins, we've added the following files:
|
||||
|
||||
### 具体文件列表
|
||||
### File Inventory
|
||||
|
||||
```
|
||||
scripts/
|
||||
├── deploy_filter.py ✨ 通用 Filter 部署工具
|
||||
├── deploy_async_context_compression.py ✨ Async Context Compression 快捷部署
|
||||
├── deploy_pipe.py (已有) Pipe 部署工具
|
||||
├── DEPLOYMENT_GUIDE.md ✨ 完整部署指南
|
||||
├── DEPLOYMENT_SUMMARY.md ✨ 部署功能总结
|
||||
├── QUICK_START.md ✨ 快速参考卡片
|
||||
├── .env (需要创建) API 密钥配置
|
||||
└── ...其他现有脚本
|
||||
├── install_all_plugins.py ✨ Batch install Action/Filter/Pipe/Tool plugins
|
||||
├── deploy_filter.py ✨ Generic Filter deployment tool
|
||||
├── deploy_tool.py ✨ Tool plugin deployment tool
|
||||
├── deploy_async_context_compression.py ✨ Async Context Compression quick deploy
|
||||
├── deploy_pipe.py (existing) Pipe deployment tool
|
||||
├── DEPLOYMENT_GUIDE.md ✨ Complete deployment guide
|
||||
├── DEPLOYMENT_SUMMARY.md ✨ Deploy feature summary
|
||||
├── QUICK_START.md ✨ Quick reference card
|
||||
├── .env (create as needed) API key configuration
|
||||
└── ...other existing scripts
|
||||
```
|
||||
|
||||
## ⚡ 快速开始 (30 秒)
|
||||
## ⚡ Quick Start (30 seconds)
|
||||
|
||||
### 步骤 1: 准备 API 密钥
|
||||
### Step 1: Prepare Your API Key
|
||||
|
||||
```bash
|
||||
cd scripts
|
||||
|
||||
# 获取你的 OpenWebUI API 密钥：
|
||||
# 1. 打开 OpenWebUI → 用户菜单 → Settings
|
||||
# 2. 找到 "API Keys" 部分
|
||||
# 3. 复制你的密钥（以 sk- 开头）
|
||||
# Get your OpenWebUI API key:
|
||||
# 1. Open OpenWebUI → User menu → Settings
|
||||
# 2. Find the "API Keys" section
|
||||
# 3. Copy your key (starts with sk-)
|
||||
|
||||
# 创建 .env 文件
|
||||
echo "api_key=sk-你的密钥" > .env
|
||||
# Create .env file
|
||||
cat > .env <<'EOF'
|
||||
api_key=sk-your-key-here
|
||||
url=http://localhost:3000
|
||||
EOF
|
||||
```
|
||||
|
||||
### 步骤 2: 部署异步上下文压缩
|
||||
### Step 2a: Install All Plugins (Recommended)
|
||||
|
||||
```bash
|
||||
# 最简单的方式 - 专用脚本
|
||||
python install_all_plugins.py
|
||||
```
|
||||
|
||||
### Step 2b: Or Deploy Individual Plugins
|
||||
|
||||
```bash
|
||||
# Easiest way - dedicated script
|
||||
python deploy_async_context_compression.py
|
||||
|
||||
# 或使用通用脚本
|
||||
# Or use generic script
|
||||
python deploy_filter.py
|
||||
|
||||
# 或指定插件名称
|
||||
# Or specify plugin name
|
||||
python deploy_filter.py async-context-compression
|
||||
|
||||
# Or deploy a Tool
|
||||
python deploy_tool.py
|
||||
```
|
||||
|
||||
## 📋 部署工具详解
|
||||
## 📋 Deployment Tools Detailed
|
||||
|
||||
### 1️⃣ `deploy_async_context_compression.py` — 专用部署脚本
|
||||
### 1️⃣ `deploy_async_context_compression.py` — Dedicated Deployment Script
|
||||
|
||||
**最简单的部署方式！**
|
||||
**The simplest way to deploy!**
|
||||
|
||||
```bash
|
||||
cd scripts
|
||||
python deploy_async_context_compression.py
|
||||
```
|
||||
|
||||
**特点**:
|
||||
- ✅ 专为 async_context_compression 优化
|
||||
- ✅ 清晰的部署步骤和确认
|
||||
- ✅ 友好的错误提示
|
||||
- ✅ 部署成功后显示后续步骤
|
||||
**Features**:
|
||||
- ✅ Optimized specifically for async_context_compression
|
||||
- ✅ Clear deployment steps and confirmation
|
||||
- ✅ Friendly error messages
|
||||
- ✅ Shows next steps after successful deployment
|
||||
|
||||
**输出样例**:
|
||||
**Sample Output**:
|
||||
```
|
||||
======================================================================
|
||||
🚀 Deploying Async Context Compression Filter Plugin
|
||||
@@ -79,269 +93,314 @@ python deploy_async_context_compression.py
|
||||
======================================================================
|
||||
|
||||
Next steps:
|
||||
1. Open OpenWebUI in your browser: http://localhost:3003
|
||||
1. Open OpenWebUI in your browser: http://localhost:3000
|
||||
2. Go to Settings → Filters
|
||||
3. Enable 'Async Context Compression'
|
||||
4. Configure Valves as needed
|
||||
5. Start using the filter in conversations
|
||||
```
|
||||
|
||||
### 2️⃣ `deploy_filter.py` — 通用 Filter 部署工具
|
||||
### 2️⃣ `deploy_filter.py` — Generic Filter Deployment Tool
|
||||
|
||||
**支持所有 Filter 插件！**
|
||||
**Supports all Filter plugins!**
|
||||
|
||||
```bash
|
||||
# 部署默认的 async_context_compression
|
||||
# Deploy default async_context_compression
|
||||
python deploy_filter.py
|
||||
|
||||
# 部署其他 Filter
|
||||
# Deploy other Filters
|
||||
python deploy_filter.py folder-memory
|
||||
python deploy_filter.py context_enhancement_filter
|
||||
|
||||
# 列出所有可用 Filter
|
||||
# List all available Filters
|
||||
python deploy_filter.py --list
|
||||
```
|
||||
|
||||
**特点**:
|
||||
- ✅ 通用的 Filter 部署工具
|
||||
- ✅ 支持多个插件
|
||||
- ✅ 自动元数据提取
|
||||
- ✅ 智能更新/创建逻辑
|
||||
- ✅ 完整的错误诊断
|
||||
**Features**:
|
||||
- ✅ Generic Filter deployment tool
|
||||
- ✅ Supports multiple plugins
|
||||
- ✅ Auto metadata extraction
|
||||
- ✅ Smart update/create logic
|
||||
- ✅ Complete error diagnostics
|
||||
|
||||
### 3️⃣ `deploy_pipe.py` — Pipe 部署工具
|
||||
### 3️⃣ `deploy_pipe.py` — Pipe Deployment Tool
|
||||
|
||||
```bash
|
||||
python deploy_pipe.py
|
||||
```
|
||||
|
||||
用于部署 Pipe 类型的插件（如 GitHub Copilot SDK）。
|
||||
Used to deploy Pipe-type plugins (like GitHub Copilot SDK).
|
||||
|
||||
## 🔧 工作原理
|
||||
|
||||
```
|
||||
你的代码变更
|
||||
↓
|
||||
运行部署脚本
|
||||
↓
|
||||
脚本读取对应插件文件
|
||||
↓
|
||||
从代码自动提取元数据 (title, version, author, etc.)
|
||||
↓
|
||||
构建 API 请求
|
||||
↓
|
||||
发送到本地 OpenWebUI
|
||||
↓
|
||||
OpenWebUI 更新或创建插件
|
||||
↓
|
||||
立即生效！（无需重启）
|
||||
```
|
||||
|
||||
## 📊 可部署的 Filter 列表
|
||||
|
||||
使用 `python deploy_filter.py --list` 查看所有可用 Filter：
|
||||
|
||||
| Filter 名称 | Python 文件 | 描述 |
|
||||
|-----------|-----------|------|
|
||||
| **async-context-compression** | async_context_compression.py | 异步上下文压缩 |
|
||||
| chat-session-mapping-filter | chat_session_mapping_filter.py | 聊天会话映射 |
|
||||
| context_enhancement_filter | context_enhancement_filter.py | 上下文增强 |
|
||||
| folder-memory | folder_memory.py | 文件夹记忆 |
|
||||
| github_copilot_sdk_files_filter | github_copilot_sdk_files_filter.py | Copilot SDK Files |
|
||||
| markdown_normalizer | markdown_normalizer.py | Markdown 规范化 |
|
||||
| web_gemini_multimodel_filter | web_gemini_multimodel_filter.py | Gemini 多模态 |
|
||||
|
||||
## 🎯 常见使用场景
|
||||
|
||||
### 场景 1: 开发新功能后部署
|
||||
### 3️⃣+ `deploy_tool.py` — Tool Deployment Tool
|
||||
|
||||
```bash
|
||||
# 1. 修改代码
|
||||
# Deploy default Tool
|
||||
python deploy_tool.py
|
||||
|
||||
# Or specify a specific Tool
|
||||
python deploy_tool.py openwebui-skills-manager
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- ✅ Supports Tools plugin deployment
|
||||
- ✅ Auto-detects `Tools` class definition
|
||||
- ✅ Smart update/create logic
|
||||
- ✅ Complete error diagnostics
|
||||
|
||||
**Use Case**:
|
||||
Deploy or reinstall a specific Tool individually, or deploy only Tools without running full batch installation. The script now calls OpenWebUI's native `/api/v1/tools/*` endpoints.
|
||||
|
||||
### 4️⃣ `install_all_plugins.py` — Batch Installation Script
|
||||
|
||||
One-command installation of all repository plugins that meet these criteria:
|
||||
|
||||
- Located in `plugins/actions`, `plugins/filters`, `plugins/pipes`, `plugins/tools`
|
||||
- Plugin header contains `openwebui_id`
|
||||
- Filename is not in Chinese characters
|
||||
- Filename does not end with `_cn.py`
|
||||
|
||||
```bash
|
||||
# Check which plugins will be installed
|
||||
python install_all_plugins.py --list
|
||||
|
||||
# Dry-run without calling API
|
||||
python install_all_plugins.py --dry-run
|
||||
|
||||
# Actually install all supported types (including Action/Filter/Pipe/Tool)
|
||||
python install_all_plugins.py
|
||||
|
||||
# Install only specific types
|
||||
python install_all_plugins.py --types pipe action
|
||||
```
|
||||
|
||||
The script prioritizes updating existing plugins and automatically creates new ones.
|
||||
|
||||
**Tool Integration**: Tool-type plugins now automatically use OpenWebUI's native `/api/v1/tools/create` and `/api/v1/tools/id/{id}/update` endpoints, no longer reusing the `functions` endpoint.
|
||||
|
||||
## 🔧 How It Works
|
||||
|
||||
```
|
||||
Your code changes
|
||||
↓
|
||||
Run deployment script
|
||||
↓
|
||||
Script reads the corresponding plugin file
|
||||
↓
|
||||
Auto-extracts metadata from code (title, version, author, etc.)
|
||||
↓
|
||||
Builds API request
|
||||
↓
|
||||
Sends to local OpenWebUI
|
||||
↓
|
||||
OpenWebUI updates or creates plugin
|
||||
↓
|
||||
Takes effect immediately! (no restart needed)
|
||||
```
|
||||
|
||||
## 📊 Available Filter List
|
||||
|
||||
Use `python deploy_filter.py --list` to see all available Filters:
|
||||
|
||||
| Filter Name | Python File | Description |
|
||||
|-----------|-----------|------|
|
||||
| **async-context-compression** | async_context_compression.py | Async context compression |
|
||||
| chat-session-mapping-filter | chat_session_mapping_filter.py | Chat session mapping |
|
||||
| context_enhancement_filter | context_enhancement_filter.py | Context enhancement |
|
||||
| folder-memory | folder_memory.py | Folder memory |
|
||||
| github_copilot_sdk_files_filter | github_copilot_sdk_files_filter.py | Copilot SDK Files |
|
||||
| markdown_normalizer | markdown_normalizer.py | Markdown normalization |
|
||||
| web_gemini_multimodel_filter | web_gemini_multimodel_filter.py | Gemini multimodal |
|
||||
|
||||
## 🎯 Common Use Cases
|
||||
|
||||
### Scenario 1: Deploy After Feature Development
|
||||
|
||||
```bash
|
||||
# 1. Modify code
|
||||
vim ../plugins/filters/async-context-compression/async_context_compression.py
|
||||
|
||||
# 2. 更新版本号（可选）
|
||||
# 2. Update version number (optional)
|
||||
# version: 1.3.0 → 1.3.1
|
||||
|
||||
# 3. 部署
|
||||
# 3. Deploy
|
||||
python deploy_async_context_compression.py
|
||||
|
||||
# 4. 在 OpenWebUI 中测试
|
||||
# → 无需重启，立即生效！
|
||||
# 4. Test in OpenWebUI
|
||||
# → No restart needed, takes effect immediately!
|
||||
|
||||
# 5. 继续开发，重复上述步骤
|
||||
# 5. Continue development and repeat
|
||||
```
|
||||
|
||||
### 场景 2: 修复 Bug 并快速验证
|
||||
### Scenario 2: Fix Bug and Verify Quickly
|
||||
|
||||
```bash
|
||||
# 1. 定位并修复 Bug
|
||||
# 1. Find and fix bug
|
||||
vim ../plugins/filters/async-context-compression/async_context_compression.py
|
||||
|
||||
# 2. 快速部署验证
|
||||
# 2. Quick deploy to verify
|
||||
python deploy_async_context_compression.py
|
||||
|
||||
# 3. 在 OpenWebUI 测试 Bug 修复
|
||||
# 一键部署，秒级反馈！
|
||||
# 3. Test bug fix in OpenWebUI
|
||||
# One-command deploy, instant feedback!
|
||||
```
|
||||
|
||||
### 场景 3: 部署多个 Filter
|
||||
### Scenario 3: Deploy Multiple Filters
|
||||
|
||||
```bash
|
||||
# 部署所有需要更新的 Filter
|
||||
# Deploy all Filters that need updates
|
||||
python deploy_filter.py async-context-compression
|
||||
python deploy_filter.py folder-memory
|
||||
python deploy_filter.py context_enhancement_filter
|
||||
```
|
||||
|
||||
## 🔐 安全提示
|
||||
## 🔐 Security Tips
|
||||
|
||||
### 管理 API 密钥
|
||||
### Manage API Keys
|
||||
|
||||
```bash
|
||||
# 1. 创建 .env（只在本地）
|
||||
# 1. Create .env (local only)
|
||||
echo "api_key=sk-your-key" > .env
|
||||
|
||||
# 2. 添加到 .gitignore（防止提交）
|
||||
# 2. Add to .gitignore (prevent commit)
|
||||
echo "scripts/.env" >> ../.gitignore
|
||||
|
||||
# 3. 验证不会被提交
|
||||
git status # 应该看不到 .env
|
||||
# 3. Verify it won't be committed
|
||||
git status # should not show .env
|
||||
|
||||
# 4. 定期轮换密钥
|
||||
# → 在 OpenWebUI Settings 中生成新密钥
|
||||
# → 更新 .env 文件
|
||||
# 4. Rotate keys regularly
|
||||
# → Generate new key in OpenWebUI Settings
|
||||
# → Update .env file
|
||||
```
|
||||
|
||||
### ✅ 安全检查清单
|
||||
### ✅ Security Checklist
|
||||
|
||||
- [ ] `.env` 文件在 `.gitignore` 中
|
||||
- [ ] 从不在代码中硬编码 API 密钥
|
||||
- [ ] 定期轮换 API 密钥
|
||||
- [ ] 仅在可信网络中使用
|
||||
- [ ] 生产环境使用 CI/CD 秘密管理
|
||||
- [ ] `.env` file is in `.gitignore`
|
||||
- [ ] Never hardcode API keys in code
|
||||
- [ ] Rotate API keys periodically
|
||||
- [ ] Use only on trusted networks
|
||||
- [ ] Use CI/CD secret management in production
|
||||
|
||||
## ❌ 故障排除
|
||||
## ❌ Troubleshooting
|
||||
|
||||
### 问题 1: "Connection error"
|
||||
### Issue 1: "Connection error"
|
||||
|
||||
```
|
||||
❌ Connection error: Could not reach OpenWebUI at localhost:3003
|
||||
❌ Connection error: Could not reach OpenWebUI at localhost:3000
|
||||
Make sure OpenWebUI is running and accessible.
|
||||
```
|
||||
|
||||
**解决方案**:
|
||||
**Solution**:
|
||||
```bash
|
||||
# 1. 检查 OpenWebUI 是否运行
|
||||
curl http://localhost:3003
|
||||
# 1. Check if OpenWebUI is running
|
||||
curl http://localhost:3000
|
||||
|
||||
# 2. 如果端口不同，编辑脚本中的 URL
|
||||
# 默认: http://localhost:3003
|
||||
# 修改位置: deploy_filter.py 中的 "localhost:3003"
|
||||
# 2. If port is different, edit URL in script
|
||||
# Default: http://localhost:3000
|
||||
# Location: "localhost:3000" in deploy_filter.py
|
||||
|
||||
# 3. 检查防火墙设置
|
||||
# 3. Check firewall settings
|
||||
```
|
||||
|
||||
### 问题 2: ".env file not found"
|
||||
### Issue 2: ".env file not found"
|
||||
|
||||
```
|
||||
❌ [ERROR] .env file not found at .env
|
||||
Please create it with: api_key=sk-xxxxxxxxxxxx
|
||||
```
|
||||
|
||||
**解决方案**:
|
||||
**Solution**:
|
||||
```bash
|
||||
echo "api_key=sk-your-api-key" > .env
|
||||
cat .env # 验证文件已创建
|
||||
cat .env # verify file created
|
||||
```
|
||||
|
||||
### 问题 3: "Filter not found"
|
||||
### Issue 3: "Filter not found"
|
||||
|
||||
```
|
||||
❌ [ERROR] Filter 'xxx' not found in .../plugins/filters
|
||||
```
|
||||
|
||||
**解决方案**:
|
||||
**Solution**:
|
||||
```bash
|
||||
# 列出所有可用 Filter
|
||||
# List all available Filters
|
||||
python deploy_filter.py --list
|
||||
|
||||
# 使用正确的名称重试
|
||||
# Retry with correct name
|
||||
python deploy_filter.py async-context-compression
|
||||
```
|
||||
|
||||
### 问题 4: "Status 401" (Unauthorized)
|
||||
### Issue 4: "Status 401" (Unauthorized)
|
||||
|
||||
```
|
||||
❌ Failed to update or create. Status: 401
|
||||
Error: {"error": "Unauthorized"}
|
||||
```
|
||||
|
||||
**解决方案**:
|
||||
**Solution**:
|
||||
```bash
|
||||
# 1. 验证 API 密钥是否正确
|
||||
# 1. Verify API key is correct
|
||||
grep "api_key=" .env
|
||||
|
||||
# 2. 在 OpenWebUI 中检查密钥是否仍然有效
|
||||
# Settings → API Keys → 检查
|
||||
# 2. Check if key is still valid in OpenWebUI
|
||||
# Settings → API Keys → Check
|
||||
|
||||
# 3. 生成新密钥并更新 .env
|
||||
# 3. Generate new key and update .env
|
||||
echo "api_key=sk-new-key" > .env
|
||||
```
|
||||
|
||||
## 📖 文档导航
|
||||
## 📖 Documentation Navigation
|
||||
|
||||
| 文档 | 描述 |
|
||||
| Document | Description |
|
||||
|------|------|
|
||||
| **README.md** (本文件) | 快速参考和常见问题 |
|
||||
| [QUICK_START.md](QUICK_START.md) | 一页速查表 |
|
||||
| [DEPLOYMENT_GUIDE.md](DEPLOYMENT_GUIDE.md) | 完整详细指南 |
|
||||
| [DEPLOYMENT_SUMMARY.md](DEPLOYMENT_SUMMARY.md) | 技术架构说明 |
|
||||
| **README.md** (this file) | Quick reference and FAQs |
|
||||
| [QUICK_START.md](QUICK_START.md) | One-page cheat sheet |
|
||||
| [DEPLOYMENT_GUIDE.md](DEPLOYMENT_GUIDE.md) | Complete detailed guide |
|
||||
| [DEPLOYMENT_SUMMARY.md](DEPLOYMENT_SUMMARY.md) | Technical architecture |
|
||||
|
||||
## 🧪 验证部署成功
|
||||
## 🧪 Verify Deployment Success
|
||||
|
||||
### 方式 1: 检查脚本输出
|
||||
### Method 1: Check Script Output
|
||||
|
||||
```bash
|
||||
python deploy_async_context_compression.py
|
||||
|
||||
# 成功标志:
|
||||
# Success indicator:
|
||||
✅ Successfully updated 'Async Context Compression' filter!
|
||||
```
|
||||
|
||||
### 方式 2: 在 OpenWebUI 中验证
|
||||
### Method 2: Verify in OpenWebUI
|
||||
|
||||
1. 打开 OpenWebUI: http://localhost:3003
|
||||
2. 进入 Settings → Filters
|
||||
3. 查看 "Async Context Compression" 是否列出
|
||||
4. 查看版本号是否正确（应该是最新的）
|
||||
1. Open OpenWebUI: http://localhost:3000
|
||||
2. Go to Settings → Filters
|
||||
3. Check if 'Async Context Compression' is listed
|
||||
4. Verify version number is correct (should be latest)
|
||||
|
||||
### 方式 3: 测试插件功能
|
||||
### Method 3: Test Plugin Functionality
|
||||
|
||||
1. 打开一个新对话
|
||||
2. 启用 "Async Context Compression" Filter
|
||||
3. 进行多轮对话，验证压缩和总结功能正常
|
||||
1. Open a new conversation
|
||||
2. Enable 'Async Context Compression' Filter
|
||||
3. Have multiple-turn conversation and verify compression/summarization works
|
||||
## 💡 Advanced Usage
|
||||
|
||||
## 💡 高级用法
|
||||
|
||||
### 自动化部署测试
|
||||
### Automated Deploy & Test
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# deploy_and_test.sh
|
||||
|
||||
echo "部署插件..."
|
||||
echo "Deploying plugin..."
|
||||
python scripts/deploy_async_context_compression.py
|
||||
|
||||
if [ $? -eq 0 ]; then
|
||||
echo "✅ 部署成功，运行测试..."
|
||||
echo "✅ Deploy successful, running tests..."
|
||||
python -m pytest tests/plugins/filters/async-context-compression/ -v
|
||||
else
|
||||
echo "❌ 部署失败"
|
||||
echo "❌ Deploy failed"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
### CI/CD 集成
|
||||
### CI/CD Integration
|
||||
|
||||
```yaml
|
||||
# .github/workflows/deploy.yml
|
||||
@@ -362,55 +421,56 @@ jobs:
|
||||
api_key: ${{ secrets.OPENWEBUI_API_KEY }}
|
||||
```
|
||||
|
||||
## 📞 获取帮助
|
||||
## 📞 Getting Help
|
||||
|
||||
### 检查脚本状态
|
||||
### Check Script Status
|
||||
|
||||
```bash
|
||||
# 列出所有可用脚本
|
||||
# List all available scripts
|
||||
ls -la scripts/*.py
|
||||
|
||||
# 检查部署脚本是否存在
|
||||
# Check if deployment scripts exist
|
||||
ls -la scripts/deploy_*.py
|
||||
```
|
||||
|
||||
### 查看脚本版本
|
||||
### View Script Help
|
||||
|
||||
```bash
|
||||
# 查看脚本帮助
|
||||
python scripts/deploy_filter.py --help # 如果支持的话
|
||||
# View help (if supported)
|
||||
python scripts/deploy_filter.py --help # if supported
|
||||
python scripts/deploy_async_context_compression.py --help
|
||||
```
|
||||
|
||||
### 调试模式
|
||||
### Debug Mode
|
||||
|
||||
```bash
|
||||
# 保存输出到日志文件
|
||||
# Save output to log file
|
||||
python scripts/deploy_async_context_compression.py | tee deploy.log
|
||||
|
||||
# 检查日志
|
||||
# Check log
|
||||
cat deploy.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 文件清单
|
||||
## 📝 File Checklist
|
||||
|
||||
新增的部署相关文件：
|
||||
Newly created deployment-related files:
|
||||
|
||||
```
|
||||
✨ scripts/deploy_filter.py (新增) ~300 行
|
||||
✨ scripts/deploy_async_context_compression.py (新增) ~70 行
|
||||
✨ scripts/DEPLOYMENT_GUIDE.md (新增) 完整指南
|
||||
✨ scripts/DEPLOYMENT_SUMMARY.md (新增) 技术总结
|
||||
✨ scripts/QUICK_START.md (新增) 快速参考
|
||||
📄 tests/scripts/test_deploy_filter.py (新增) 10 个单元测试 ✅
|
||||
✨ scripts/deploy_filter.py (new) ~300 lines
|
||||
✨ scripts/deploy_async_context_compression.py (new) ~70 lines
|
||||
✨ scripts/DEPLOYMENT_GUIDE.md (new) complete guide
|
||||
✨ scripts/DEPLOYMENT_SUMMARY.md (new) technical summary
|
||||
✨ scripts/QUICK_START.md (new) quick reference
|
||||
📄 tests/scripts/test_deploy_filter.py (new) 10 unit tests ✅
|
||||
|
||||
✅ 所有文件已创建并测试通过！
|
||||
✅ All files created and tested successfully!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**最后更新**: 2026-03-09
|
||||
**脚本状态**: ✅ Ready for production
|
||||
**测试覆盖**: 10/10 通过 ✅
|
||||
**Last Updated**: 2026-03-09
|
||||
**Script Status**: ✅ Ready for production
|
||||
**Test Coverage**: 10/10 passed ✅
|
||||
|
||||
|
||||
@@ -1,45 +1,45 @@
|
||||
# 🔄 部署脚本的更新机制 (Deployment Update Mechanism)
|
||||
# 🔄 Deployment Scripts Update Mechanism
|
||||
|
||||
## 核心答案
|
||||
## Core Answer
|
||||
|
||||
✅ **是的，再次部署会自动更新！**
|
||||
✅ **Yes, re-deploying automatically updates the plugin!**
|
||||
|
||||
部署脚本采用**智能两阶段策略**：
|
||||
1. 🔄 **优先尝试更新** (UPDATE) — 如果插件已存在
|
||||
2. 📝 **自动创建** (CREATE) — 如果更新失败（插件不存在）
|
||||
The deployment script uses a **smart two-stage strategy**:
|
||||
1. 🔄 **Try UPDATE First** (if plugin exists)
|
||||
2. 📝 **Auto CREATE** (if update fails — plugin doesn't exist)
|
||||
|
||||
## 工作流程图
|
||||
## Workflow Diagram
|
||||
|
||||
```
|
||||
运行部署脚本
|
||||
Run deploy script
|
||||
↓
|
||||
读取本地代码和元数据
|
||||
Read local code and metadata
|
||||
↓
|
||||
发送 UPDATE 请求到 OpenWebUI
|
||||
Send UPDATE request to OpenWebUI
|
||||
↓
|
||||
├─ HTTP 200 ✅
|
||||
│ └─ 插件已存在 → 更新成功！
|
||||
│ └─ Plugin exists → Update successful!
|
||||
│
|
||||
└─ 其他状态代码 (404, 400 等)
|
||||
└─ 插件不存在或更新失败
|
||||
└─ Other status codes (404, 400, etc.)
|
||||
└─ Plugin doesn't exist or update failed
|
||||
↓
|
||||
发送 CREATE 请求
|
||||
Send CREATE request
|
||||
↓
|
||||
├─ HTTP 200 ✅
|
||||
│ └─ 创建成功！
|
||||
│ └─ Creation successful!
|
||||
│
|
||||
└─ 失败
|
||||
└─ 显示错误信息
|
||||
└─ Failed
|
||||
└─ Display error message
|
||||
```
|
||||
|
||||
## 详细步骤分析
|
||||
## Detailed Step-by-step
|
||||
|
||||
### 步骤 1️⃣: 尝试更新 (UPDATE)
|
||||
### Step 1️⃣: Try UPDATE First
|
||||
|
||||
```python
|
||||
# 代码位置: deploy_filter.py 第 220-230 行
|
||||
# Code location: deploy_filter.py line 220-230
|
||||
|
||||
update_url = "http://localhost:3003/api/v1/functions/id/{filter_id}/update"
|
||||
update_url = "http://localhost:3000/api/v1/functions/id/{filter_id}/update"
|
||||
|
||||
response = requests.post(
|
||||
update_url,
|
||||
@@ -53,24 +53,24 @@ if response.status_code == 200:
|
||||
return True
|
||||
```
|
||||
|
||||
**这一步**:
|
||||
- 向 OpenWebUI API 发送 **POST** 到 `/api/v1/functions/id/{filter_id}/update`
|
||||
- 如果返回 **HTTP 200**，说明插件已存在且成功更新
|
||||
- 包含的内容:
|
||||
- 完整的最新代码
|
||||
- 元数据 (title, version, author, description 等)
|
||||
- 清单信息 (manifest)
|
||||
**What Happens**:
|
||||
- Send **POST** to `/api/v1/functions/id/{filter_id}/update`
|
||||
- If returns **HTTP 200**, plugin exists and update succeeded
|
||||
- Includes:
|
||||
- Complete latest code
|
||||
- Metadata (title, version, author, description, etc.)
|
||||
- Manifest information
|
||||
|
||||
### 步骤 2️⃣: 若更新失败，尝试创建 (CREATE)
|
||||
### Step 2️⃣: If UPDATE Fails, Try CREATE
|
||||
|
||||
```python
|
||||
# 代码位置: deploy_filter.py 第 231-245 行
|
||||
# Code location: deploy_filter.py line 231-245
|
||||
|
||||
if response.status_code != 200:
|
||||
print(f"⚠️ Update failed with status {response.status_code}, "
|
||||
"attempting to create instead...")
|
||||
|
||||
create_url = "http://localhost:3003/api/v1/functions/create"
|
||||
create_url = "http://localhost:3000/api/v1/functions/create"
|
||||
res_create = requests.post(
|
||||
create_url,
|
||||
headers=headers,
|
||||
@@ -83,96 +83,96 @@ if response.status_code != 200:
|
||||
return True
|
||||
```
|
||||
|
||||
**这一步**:
|
||||
- 如果更新失败 (HTTP ≠ 200)，自动尝试创建
|
||||
- 向 `/api/v1/functions/create` 发送 **POST** 请求
|
||||
- 使用**相同的 payload**（代码、元数据都一样）
|
||||
- 如果创建成功，第一次部署到 OpenWebUI
|
||||
**What Happens**:
|
||||
- If update fails (HTTP ≠ 200), auto-attempt create
|
||||
- Send **POST** to `/api/v1/functions/create`
|
||||
- Uses **same payload** (code, metadata identical)
|
||||
- If creation succeeds, first deployment to OpenWebUI
|
||||
|
||||
## 实际使用场景
|
||||
## Real-world Scenarios
|
||||
|
||||
### 场景 A: 第一次部署
|
||||
### Scenario A: First Deployment
|
||||
|
||||
```bash
|
||||
$ python deploy_async_context_compression.py
|
||||
|
||||
📦 Deploying filter 'Async Context Compression' (version 1.3.0)...
|
||||
File: .../async_context_compression.py
|
||||
⚠️ Update failed with status 404, attempting to create instead... ← 第一次，插件不存在
|
||||
✅ Successfully created 'Async Context Compression' filter! ← 创建成功
|
||||
⚠️ Update failed with status 404, attempting to create instead... ← First time, plugin doesn't exist
|
||||
✅ Successfully created 'Async Context Compression' filter! ← Creation succeeds
|
||||
```
|
||||
|
||||
**发生的事**:
|
||||
1. 尝试 UPDATE → 失败 (HTTP 404 — 插件不存在)
|
||||
2. 自动尝试 CREATE → 成功 (HTTP 200)
|
||||
3. 插件被创建到 OpenWebUI
|
||||
**What Happens**:
|
||||
1. Try UPDATE → fails (HTTP 404 — plugin doesn't exist)
|
||||
2. Auto-try CREATE → succeeds (HTTP 200)
|
||||
3. Plugin created in OpenWebUI
|
||||
|
||||
---
|
||||
|
||||
### 场景 B: 再次部署 (修改代码后)
|
||||
### Scenario B: Re-deploy After Code Changes
|
||||
|
||||
```bash
|
||||
# 第一次修改代码，再次部署
|
||||
# Made first code change, deploying again
|
||||
$ python deploy_async_context_compression.py
|
||||
|
||||
📦 Deploying filter 'Async Context Compression' (version 1.3.1)...
|
||||
File: .../async_context_compression.py
|
||||
✅ Successfully updated 'Async Context Compression' filter! ← 直接更新！
|
||||
✅ Successfully updated 'Async Context Compression' filter! ← Direct update!
|
||||
```
|
||||
|
||||
**发生的事**:
|
||||
1. 读取修改后的代码
|
||||
2. 尝试 UPDATE → 成功 (HTTP 200 — 插件已存在)
|
||||
3. OpenWebUI 中的插件被更新为最新代码
|
||||
4. **无需重启 OpenWebUI**，立即生效！
|
||||
**What Happens**:
|
||||
1. Read modified code
|
||||
2. Try UPDATE → succeeds (HTTP 200 — plugin exists)
|
||||
3. Plugin in OpenWebUI updated to latest code
|
||||
4. **No need to restart OpenWebUI**, takes effect immediately!
|
||||
|
||||
---
|
||||
|
||||
### 场景 C: 多次快速迭代
|
||||
### Scenario C: Multiple Fast Iterations
|
||||
|
||||
```bash
|
||||
# 第1次修改
|
||||
# 1st change
|
||||
$ python deploy_async_context_compression.py
|
||||
✅ Successfully updated 'Async Context Compression' filter!
|
||||
|
||||
# 第2次修改
|
||||
# 2nd change
|
||||
$ python deploy_async_context_compression.py
|
||||
✅ Successfully updated 'Async Context Compression' filter!
|
||||
|
||||
# 第3次修改
|
||||
# 3rd change
|
||||
$ python deploy_async_context_compression.py
|
||||
✅ Successfully updated 'Async Context Compression' filter!
|
||||
|
||||
# ... 无限制地重复 ...
|
||||
# ... repeat infinitely ...
|
||||
```
|
||||
|
||||
**特点**:
|
||||
- 🚀 每次更新只需 5 秒
|
||||
- 📝 每次都是增量更新
|
||||
- ✅ 无需重启 OpenWebUI
|
||||
- 🔄 可以无限制地重复
|
||||
**Characteristics**:
|
||||
- 🚀 Each update takes only 5 seconds
|
||||
- 📝 Each is an incremental update
|
||||
- ✅ No need to restart OpenWebUI
|
||||
- 🔄 Can repeat indefinitely
|
||||
|
||||
## 更新的内容清单
|
||||
## What Gets Updated
|
||||
|
||||
每次部署时，以下内容会被更新：
|
||||
Each deployment updates the following:
|
||||
|
||||
✅ **代码** — 全部最新的 Python 代码
|
||||
✅ **版本号** — 从 docstring 自动提取
|
||||
✅ **标题** — 插件的显示名称
|
||||
✅ **作者信息** — author, author_url
|
||||
✅ **描述** — plugin description
|
||||
✅ **元数据** — funding_url, openwebui_id 等
|
||||
✅ **Code** — All latest Python code
|
||||
✅ **Version** — Auto-extracted from docstring
|
||||
✅ **Title** — Plugin display name
|
||||
✅ **Author Info** — author, author_url
|
||||
✅ **Description** — Plugin description
|
||||
✅ **Metadata** — funding_url, openwebui_id, etc.
|
||||
|
||||
❌ **配置不会被覆盖** — 用户在 OpenWebUI 中设置的 Valves 配置保持不变
|
||||
❌ **Configuration NOT Overwritten** — User's Valves settings in OpenWebUI stay unchanged
|
||||
|
||||
## 版本号管理
|
||||
## Version Number Management
|
||||
|
||||
### 更新时版本号会变吗？
|
||||
### Does Version Change on Update?
|
||||
|
||||
✅ **是的，会变！**
|
||||
✅ **Yes!**
|
||||
|
||||
```python
|
||||
# async_context_compression.py 的 docstring
|
||||
# docstring in async_context_compression.py
|
||||
|
||||
"""
|
||||
title: Async Context Compression
|
||||
@@ -180,124 +180,124 @@ version: 1.3.0
|
||||
"""
|
||||
```
|
||||
|
||||
**每次部署时**:
|
||||
1. 脚本从 docstring 读取版本号
|
||||
2. 发送给 OpenWebUI 的 manifest 包含这个版本号
|
||||
3. 如果代码中改了版本号，部署时会更新到新版本
|
||||
**Each deployment**:
|
||||
1. Script reads version from docstring
|
||||
2. Sends this version in manifest to OpenWebUI
|
||||
3. If you change version in code, deployment updates to new version
|
||||
|
||||
**最佳实践**:
|
||||
**Best Practice**:
|
||||
```bash
|
||||
# 1. 修改代码
|
||||
# 1. Modify code
|
||||
vim async_context_compression.py
|
||||
|
||||
# 2. 更新版本号（在 docstring 中）
|
||||
# 版本: 1.3.0 → 1.3.1
|
||||
# 2. Update version (in docstring)
|
||||
# version: 1.3.0 → 1.3.1
|
||||
|
||||
# 3. 部署
|
||||
# 3. Deploy
|
||||
python deploy_async_context_compression.py
|
||||
|
||||
# 结果: OpenWebUI 中显示版本 1.3.1
|
||||
# Result: OpenWebUI shows version 1.3.1
|
||||
```
|
||||
|
||||
## 部署失败的情况
|
||||
## Deployment Failure Cases
|
||||
|
||||
### 情况 1: 网络错误
|
||||
### Case 1: Network Error
|
||||
|
||||
```bash
|
||||
❌ Connection error: Could not reach OpenWebUI at localhost:3003
|
||||
❌ Connection error: Could not reach OpenWebUI at localhost:3000
|
||||
Make sure OpenWebUI is running and accessible.
|
||||
```
|
||||
|
||||
**原因**: OpenWebUI 未运行或端口错误
|
||||
**解决**: 检查 OpenWebUI 是否在运行
|
||||
**Cause**: OpenWebUI not running or wrong port
|
||||
**Solution**: Check if OpenWebUI is running
|
||||
|
||||
### 情况 2: API 密钥无效
|
||||
### Case 2: Invalid API Key
|
||||
|
||||
```bash
|
||||
❌ Failed to update or create. Status: 401
|
||||
Error: {"error": "Unauthorized"}
|
||||
```
|
||||
|
||||
**原因**: .env 中的 API 密钥无效或过期
|
||||
**解决**: 更新 `.env` 文件中的 api_key
|
||||
**Cause**: API key in .env is invalid or expired
|
||||
**Solution**: Update api_key in `.env` file
|
||||
|
||||
### 情况 3: 服务器错误
|
||||
### Case 3: Server Error
|
||||
|
||||
```bash
|
||||
❌ Failed to update or create. Status: 500
|
||||
Error: Internal server error
|
||||
```
|
||||
|
||||
**原因**: OpenWebUI 服务器内部错误
|
||||
**解决**: 检查 OpenWebUI 日志
|
||||
**Cause**: OpenWebUI server internal error
|
||||
**Solution**: Check OpenWebUI logs
|
||||
|
||||
## 设置版本号的最佳实践
|
||||
## Setting Version Numbers — Best Practices
|
||||
|
||||
### 语义化版本 (Semantic Versioning)
|
||||
### Semantic Versioning
|
||||
|
||||
遵循 `MAJOR.MINOR.PATCH` 格式：
|
||||
Follow `MAJOR.MINOR.PATCH` format:
|
||||
|
||||
```python
|
||||
"""
|
||||
version: 1.3.0
|
||||
│ │ │
|
||||
│ │ └─ PATCH: Bug 修复 (1.3.0 → 1.3.1)
|
||||
│ └────── MINOR: 新功能 (1.3.0 → 1.4.0)
|
||||
└───────── MAJOR: 破坏性变更 (1.3.0 → 2.0.0)
|
||||
│ │ └─ PATCH: Bug fixes (1.3.0 → 1.3.1)
|
||||
│ └────── MINOR: New features (1.3.0 → 1.4.0)
|
||||
└───────── MAJOR: Breaking changes (1.3.0 → 2.0.0)
|
||||
"""
|
||||
```
|
||||
|
||||
**例子**:
|
||||
**Examples**:
|
||||
|
||||
```python
|
||||
# Bug 修复 (PATCH)
|
||||
# Bug fix (PATCH)
|
||||
version: 1.3.0 → 1.3.1
|
||||
|
||||
# 新功能 (MINOR)
|
||||
# New feature (MINOR)
|
||||
version: 1.3.0 → 1.4.0
|
||||
|
||||
# 重大更新 (MAJOR)
|
||||
# Major update (MAJOR)
|
||||
version: 1.3.0 → 2.0.0
|
||||
```
|
||||
|
||||
## 完整的迭代工作流
|
||||
## Complete Iteration Workflow
|
||||
|
||||
```bash
|
||||
# 1. 首次部署
|
||||
# 1. First deployment
|
||||
cd scripts
|
||||
python deploy_async_context_compression.py
|
||||
# 结果: 创建插件 (第一次)
|
||||
# Result: Plugin created (first time)
|
||||
|
||||
# 2. 修改代码
|
||||
# 2. Modify code
|
||||
vim ../plugins/filters/async-context-compression/async_context_compression.py
|
||||
# 修改内容...
|
||||
# Edit code...
|
||||
|
||||
# 3. 再次部署 (自动更新)
|
||||
# 3. Deploy again (auto-update)
|
||||
python deploy_async_context_compression.py
|
||||
# 结果: 更新插件 (立即生效，无需重启 OpenWebUI)
|
||||
# Result: Plugin updated (takes effect immediately, no OpenWebUI restart)
|
||||
|
||||
# 4. 重复步骤 2-3，无限次迭代
|
||||
# 每次修改 → 每次部署 → 立即测试 → 继续改进
|
||||
# 4. Repeat steps 2-3 indefinitely
|
||||
# Modify → Deploy → Test → Improve → Repeat
|
||||
```
|
||||
|
||||
## 自动更新的优势
|
||||
## Benefits of Auto-update
|
||||
|
||||
| 优势 | 说明 |
|
||||
|-----|------|
|
||||
| ⚡ **快速迭代** | 修改代码 → 部署 (5秒) → 测试，无需等待 |
|
||||
| 🔄 **自动检测** | 无需手动判断是创建还是更新 |
|
||||
| 📝 **版本管理** | 版本号自动从代码提取 |
|
||||
| ✅ **无需重启** | OpenWebUI 无需重启，配置保持不变 |
|
||||
| 🛡️ **安全更新** | 用户配置 (Valves) 不会被覆盖 |
|
||||
| Benefit | Details |
|
||||
|---------|---------|
|
||||
| ⚡ **Fast Iteration** | Code change → Deploy (5s) → Test, no waiting |
|
||||
| 🔄 **Auto-detection** | No manual decision between create/update |
|
||||
| 📝 **Version Management** | Version auto-extracted from code |
|
||||
| ✅ **No Restart Needed** | OpenWebUI runs continuously, config stays same |
|
||||
| 🛡️ **Safe Updates** | User settings (Valves) never overwritten |
|
||||
|
||||
## 禁用自动更新? ❌
|
||||
## Disable Auto-update? ❌
|
||||
|
||||
通常**不需要**禁用自动更新，因为：
|
||||
Usually **not needed** because:
|
||||
|
||||
1. ✅ 更新是幂等的 (多次更新相同代码 = 无变化)
|
||||
2. ✅ 用户配置不会被修改
|
||||
3. ✅ 版本号自动管理
|
||||
4. ✅ 失败时自动回退
|
||||
1. ✅ Updates are idempotent (same code deployed multiple times = no change)
|
||||
2. ✅ User configuration not modified
|
||||
3. ✅ Version numbers auto-managed
|
||||
4. ✅ Failures auto-rollback
|
||||
|
||||
但如果真的需要控制，可以：
|
||||
- 手动修改脚本 (修改 `deploy_filter.py`)
|
||||
|
||||
@@ -1,91 +1,91 @@
|
||||
# 🔄 快速参考：部署更新机制 (Quick Reference)
|
||||
# 🔄 Quick Reference: Deployment Update Mechanism
|
||||
|
||||
## 最简短的答案
|
||||
## The Shortest Answer
|
||||
|
||||
✅ **再次部署会自动更新。**
|
||||
✅ **Re-deploying automatically updates the plugin.**
|
||||
|
||||
## 工作原理 (30 秒理解)
|
||||
## How It Works (30-second understanding)
|
||||
|
||||
```
|
||||
每次运行部署脚本：
|
||||
1. 优先尝试 UPDATE（如果插件已存在）→ 更新成功
|
||||
2. 失败时自动 CREATE（第一次部署时）→ 创建成功
|
||||
Each time you run the deploy script:
|
||||
1. Priority: try UPDATE (if plugin exists) → succeeds
|
||||
2. Fallback: auto CREATE (first deployment) → succeeds
|
||||
|
||||
结果：
|
||||
✅ 不管第几次部署，脚本都能正确处理
|
||||
✅ 无需手动判断创建还是更新
|
||||
✅ 立即生效，无需重启
|
||||
Result:
|
||||
✅ Works correctly every time, regardless of deployment count
|
||||
✅ No manual judgement needed between create vs update
|
||||
✅ Takes effect immediately, no restart needed
|
||||
```
|
||||
|
||||
## 三个场景
|
||||
## Three Scenarios
|
||||
|
||||
| 场景 | 发生什么 | 结果 |
|
||||
|------|---------|------|
|
||||
| **第1次部署** | UPDATE 失败 → CREATE 成功 | ✅ 插件被创建 |
|
||||
| **修改代码后再次部署** | UPDATE 直接成功 | ✅ 插件立即更新 |
|
||||
| **未修改，重复部署** | UPDATE 成功 (无任何变化) | ✅ 无效果 (安全) |
|
||||
| Scenario | What Happens | Result |
|
||||
|----------|-------------|--------|
|
||||
| **First deployment** | UPDATE fails → CREATE succeeds | ✅ Plugin created |
|
||||
| **Deploy after code change** | UPDATE succeeds directly | ✅ Plugin updates instantly |
|
||||
| **Deploy without changes** | UPDATE succeeds (no change) | ✅ Safe (no effect) |
|
||||
|
||||
## 开发流程
|
||||
## Development Workflow
|
||||
|
||||
```bash
|
||||
# 1. 第一次部署
|
||||
# 1. First deployment
|
||||
python deploy_async_context_compression.py
|
||||
# 结果: ✅ Created
|
||||
# Result: ✅ Created
|
||||
|
||||
# 2. 修改代码
|
||||
# 2. Modify code
|
||||
vim ../plugins/filters/async-context-compression/async_context_compression.py
|
||||
# 编辑...
|
||||
# Edit...
|
||||
|
||||
# 3. 再次部署 (自动更新)
|
||||
# 3. Deploy again (auto-update)
|
||||
python deploy_async_context_compression.py
|
||||
# 结果: ✅ Updated
|
||||
# Result: ✅ Updated
|
||||
|
||||
# 4. 继续修改，重复部署
|
||||
# ... 可以无限重复 ...
|
||||
# 4. Continue editing and redeploying
|
||||
# ... can repeat infinitely ...
|
||||
```
|
||||
|
||||
## 关键点
|
||||
## Key Points
|
||||
|
||||
✅ **自动化** — 不用管是更新还是创建
|
||||
✅ **快速** — 每次部署 5 秒
|
||||
✅ **安全** — 用户配置不会被覆盖
|
||||
✅ **即时** — 无需重启 OpenWebUI
|
||||
✅ **版本管理** — 自动从代码提取版本号
|
||||
✅ **Automated** — No need to worry about create vs update
|
||||
✅ **Fast** — Each deployment takes 5 seconds
|
||||
✅ **Safe** — User configuration never gets overwritten
|
||||
✅ **Instant** — No need to restart OpenWebUI
|
||||
✅ **Version Management** — Auto-extracted from code
|
||||
|
||||
## 版本号怎么管理？
|
||||
## How to Manage Version Numbers?
|
||||
|
||||
修改代码中的版本号：
|
||||
Modify the version in your code:
|
||||
|
||||
```python
|
||||
# async_context_compression.py
|
||||
|
||||
"""
|
||||
version: 1.3.0 → 1.3.1 (修复 Bug)
|
||||
version: 1.3.0 → 1.4.0 (新功能)
|
||||
version: 1.3.0 → 2.0.0 (重大更新)
|
||||
version: 1.3.0 → 1.3.1 (Bug fixes)
|
||||
version: 1.3.0 → 1.4.0 (New features)
|
||||
version: 1.3.0 → 2.0.0 (Major updates)
|
||||
"""
|
||||
```
|
||||
|
||||
然后部署，脚本会自动读取新版本号并更新。
|
||||
Then deploy, the script will auto-read the new version and update.
|
||||
|
||||
## 常见问题速答
|
||||
## Quick Q&A
|
||||
|
||||
**Q: 用户的配置会被覆盖吗？**
|
||||
A: ❌ 不会，Valves 配置保持不变
|
||||
**Q: Will user configuration be overwritten?**
|
||||
A: ❌ No, Valves configuration stays the same
|
||||
|
||||
**Q: 需要重启 OpenWebUI 吗？**
|
||||
A: ❌ 不需要，立即生效
|
||||
**Q: Do I need to restart OpenWebUI?**
|
||||
A: ❌ No, takes effect immediately
|
||||
|
||||
**Q: 更新失败了会怎样？**
|
||||
A: ✅ 安全，保持原有插件不变
|
||||
**Q: What if update fails?**
|
||||
A: ✅ Safe, keeps original plugin intact
|
||||
|
||||
**Q: 可以无限制地重复部署吗？**
|
||||
A: ✅ 可以，完全幂等
|
||||
**Q: Can I deploy unlimited times?**
|
||||
A: ✅ Yes, completely idempotent
|
||||
|
||||
## 一行总结
|
||||
## One-liner Summary
|
||||
|
||||
> 首次部署创建插件，之后每次部署自动更新，5 秒即时反馈，无需重启。
|
||||
> First deployment creates plugin, subsequent deployments auto-update, 5-second feedback, no restart needed.
|
||||
|
||||
---
|
||||
|
||||
📖 详细文档：`scripts/UPDATE_MECHANISM.md`
|
||||
📖 Full docs: `scripts/UPDATE_MECHANISM.md`
|
||||
|
||||
@@ -12,7 +12,7 @@ To get started:
|
||||
1. Create .env file with your OpenWebUI API key:
|
||||
echo "api_key=sk-your-key-here" > .env
|
||||
|
||||
2. Make sure OpenWebUI is running on localhost:3003
|
||||
2. Make sure OpenWebUI is running on localhost:3000
|
||||
|
||||
3. Run this script:
|
||||
python deploy_async_context_compression.py
|
||||
@@ -45,7 +45,7 @@ def main():
|
||||
print("=" * 70)
|
||||
print()
|
||||
print("Next steps:")
|
||||
print(" 1. Open OpenWebUI in your browser: http://localhost:3003")
|
||||
print(" 1. Open OpenWebUI in your browser: http://localhost:3000")
|
||||
print(" 2. Go to Settings → Filters")
|
||||
print(" 3. Enable 'Async Context Compression'")
|
||||
print(" 4. Configure Valves as needed")
|
||||
@@ -58,7 +58,7 @@ def main():
|
||||
print("=" * 70)
|
||||
print()
|
||||
print("Troubleshooting:")
|
||||
print(" • Check that OpenWebUI is running: http://localhost:3003")
|
||||
print(" • Check that OpenWebUI is running: http://localhost:3000")
|
||||
print(" • Verify API key in .env file")
|
||||
print(" • Check network connectivity")
|
||||
print()
|
||||
|
||||
@@ -211,8 +211,8 @@ def deploy_filter(filter_name: str = DEFAULT_FILTER) -> bool:
|
||||
}
|
||||
|
||||
# 6. Send update request
|
||||
update_url = "http://localhost:3003/api/v1/functions/id/{}/update".format(filter_id)
|
||||
create_url = "http://localhost:3003/api/v1/functions/create"
|
||||
update_url = "http://localhost:3000/api/v1/functions/id/{}/update".format(filter_id)
|
||||
create_url = "http://localhost:3000/api/v1/functions/create"
|
||||
|
||||
print(f"📦 Deploying filter '{title}' (version {version})...")
|
||||
print(f" File: {file_path}")
|
||||
@@ -257,7 +257,7 @@ def deploy_filter(filter_name: str = DEFAULT_FILTER) -> bool:
|
||||
|
||||
except requests.exceptions.ConnectionError:
|
||||
print(
|
||||
"❌ Connection error: Could not reach OpenWebUI at localhost:3003"
|
||||
"❌ Connection error: Could not reach OpenWebUI at localhost:3000"
|
||||
)
|
||||
print(" Make sure OpenWebUI is running and accessible.")
|
||||
return False
|
||||
|
||||
@@ -9,7 +9,7 @@ SCRIPT_DIR = Path(__file__).parent
|
||||
ENV_FILE = SCRIPT_DIR / ".env"
|
||||
|
||||
URL = (
|
||||
"http://localhost:3003/api/v1/functions/id/github_copilot_official_sdk_pipe/update"
|
||||
"http://localhost:3000/api/v1/functions/id/github_copilot_official_sdk_pipe/update"
|
||||
)
|
||||
FILE_PATH = SCRIPT_DIR.parent / "plugins/pipes/github-copilot-sdk/github_copilot_sdk.py"
|
||||
|
||||
@@ -103,7 +103,7 @@ def deploy_pipe() -> None:
|
||||
print(
|
||||
f"⚠️ Update failed with status {response.status_code}, attempting to create instead..."
|
||||
)
|
||||
CREATE_URL = "http://localhost:3003/api/v1/functions/create"
|
||||
CREATE_URL = "http://localhost:3000/api/v1/functions/create"
|
||||
res_create = requests.post(
|
||||
CREATE_URL, headers=headers, data=json.dumps(payload)
|
||||
)
|
||||
|
||||
322
scripts/deploy_tool.py
Normal file
322
scripts/deploy_tool.py
Normal file
@@ -0,0 +1,322 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Deploy Tools plugins to OpenWebUI instance.
|
||||
|
||||
This script deploys tool plugins to a running OpenWebUI instance.
|
||||
It reads the plugin metadata and submits it to the local API.
|
||||
|
||||
Usage:
|
||||
python deploy_tool.py # Deploy OpenWebUI Skills Manager Tool
|
||||
python deploy_tool.py <tool_name> # Deploy specific tool
|
||||
python deploy_tool.py --list # List available tools
|
||||
"""
|
||||
|
||||
import requests
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Optional, Dict, Any
|
||||
|
||||
# ─── Configuration ───────────────────────────────────────────────────────────
|
||||
SCRIPT_DIR = Path(__file__).parent
|
||||
ENV_FILE = SCRIPT_DIR / ".env"
|
||||
TOOLS_DIR = SCRIPT_DIR.parent / "plugins/tools"
|
||||
|
||||
# Default target tool
|
||||
DEFAULT_TOOL = "openwebui-skills-manager"
|
||||
|
||||
|
||||
def _load_api_key() -> str:
|
||||
"""Load API key from .env file in the same directory as this script."""
|
||||
env_values = {}
|
||||
if ENV_FILE.exists():
|
||||
for line in ENV_FILE.read_text(encoding="utf-8").splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, value = line.split("=", 1)
|
||||
env_values[key.strip().lower()] = value.strip().strip('"').strip("'")
|
||||
|
||||
api_key = (
|
||||
os.getenv("OPENWEBUI_API_KEY")
|
||||
or os.getenv("api_key")
|
||||
or env_values.get("api_key")
|
||||
or env_values.get("openwebui_api_key")
|
||||
)
|
||||
|
||||
if not api_key:
|
||||
raise ValueError(
|
||||
f"Missing api_key. Please create {ENV_FILE} with: "
|
||||
"api_key=sk-xxxxxxxxxxxx"
|
||||
)
|
||||
return api_key
|
||||
|
||||
|
||||
def _get_base_url() -> str:
|
||||
"""Load base URL from .env file or environment."""
|
||||
env_values = {}
|
||||
if ENV_FILE.exists():
|
||||
for line in ENV_FILE.read_text(encoding="utf-8").splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, value = line.split("=", 1)
|
||||
env_values[key.strip().lower()] = value.strip().strip('"').strip("'")
|
||||
|
||||
base_url = (
|
||||
os.getenv("OPENWEBUI_URL")
|
||||
or os.getenv("OPENWEBUI_BASE_URL")
|
||||
or os.getenv("url")
|
||||
or env_values.get("url")
|
||||
or env_values.get("openwebui_url")
|
||||
or env_values.get("openwebui_base_url")
|
||||
)
|
||||
|
||||
if not base_url:
|
||||
raise ValueError(
|
||||
f"Missing url. Please create {ENV_FILE} with: "
|
||||
"url=http://localhost:3000"
|
||||
)
|
||||
return base_url.rstrip("/")
|
||||
|
||||
|
||||
def _find_tool_file(tool_name: str) -> Optional[Path]:
|
||||
"""Find the main Python file for a tool.
|
||||
|
||||
Args:
|
||||
tool_name: Directory name of the tool (e.g., 'openwebui-skills-manager')
|
||||
|
||||
Returns:
|
||||
Path to the main Python file, or None if not found.
|
||||
"""
|
||||
tool_dir = TOOLS_DIR / tool_name
|
||||
if not tool_dir.exists():
|
||||
return None
|
||||
|
||||
# Try to find a .py file matching the tool name
|
||||
py_files = list(tool_dir.glob("*.py"))
|
||||
|
||||
# Prefer a file with the tool name (with hyphens converted to underscores)
|
||||
preferred_name = tool_name.replace("-", "_") + ".py"
|
||||
for py_file in py_files:
|
||||
if py_file.name == preferred_name:
|
||||
return py_file
|
||||
|
||||
# Otherwise, return the first .py file (usually the only one)
|
||||
if py_files:
|
||||
return py_files[0]
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _extract_metadata(content: str) -> Dict[str, Any]:
|
||||
"""Extract metadata from the plugin docstring."""
|
||||
metadata = {}
|
||||
|
||||
# Extract docstring
|
||||
match = re.search(r'"""(.*?)"""', content, re.DOTALL)
|
||||
if not match:
|
||||
return metadata
|
||||
|
||||
docstring = match.group(1)
|
||||
|
||||
# Extract key-value pairs
|
||||
for line in docstring.split("\n"):
|
||||
line = line.strip()
|
||||
if ":" in line and not line.startswith("#") and not line.startswith("═"):
|
||||
parts = line.split(":", 1)
|
||||
key = parts[0].strip().lower()
|
||||
value = parts[1].strip()
|
||||
metadata[key] = value
|
||||
|
||||
return metadata
|
||||
|
||||
|
||||
def _build_tool_payload(
|
||||
tool_name: str, file_path: Path, content: str, metadata: Dict[str, Any]
|
||||
) -> Dict[str, Any]:
|
||||
"""Build the payload for the tool update/create API."""
|
||||
tool_id = metadata.get("id", tool_name).replace("-", "_")
|
||||
title = metadata.get("title", tool_name)
|
||||
author = metadata.get("author", "Fu-Jie")
|
||||
author_url = metadata.get("author_url", "https://github.com/Fu-Jie/openwebui-extensions")
|
||||
funding_url = metadata.get("funding_url", "https://github.com/open-webui")
|
||||
description = metadata.get("description", f"Tool plugin: {title}")
|
||||
version = metadata.get("version", "1.0.0")
|
||||
openwebui_id = metadata.get("openwebui_id", "")
|
||||
|
||||
payload = {
|
||||
"id": tool_id,
|
||||
"name": title,
|
||||
"meta": {
|
||||
"description": description,
|
||||
"manifest": {
|
||||
"title": title,
|
||||
"author": author,
|
||||
"author_url": author_url,
|
||||
"funding_url": funding_url,
|
||||
"description": description,
|
||||
"version": version,
|
||||
"type": "tool",
|
||||
},
|
||||
"type": "tool",
|
||||
},
|
||||
"content": content,
|
||||
}
|
||||
|
||||
# Add openwebui_id if available
|
||||
if openwebui_id:
|
||||
payload["meta"]["manifest"]["openwebui_id"] = openwebui_id
|
||||
|
||||
return payload
|
||||
|
||||
|
||||
def deploy_tool(tool_name: str = DEFAULT_TOOL) -> bool:
|
||||
"""Deploy a tool plugin to OpenWebUI.
|
||||
|
||||
Args:
|
||||
tool_name: Directory name of the tool to deploy
|
||||
|
||||
Returns:
|
||||
True if successful, False otherwise
|
||||
"""
|
||||
# 1. Load API key and base URL
|
||||
try:
|
||||
api_key = _load_api_key()
|
||||
base_url = _get_base_url()
|
||||
except ValueError as e:
|
||||
print(f"[ERROR] {e}")
|
||||
return False
|
||||
|
||||
# 2. Find tool file
|
||||
file_path = _find_tool_file(tool_name)
|
||||
if not file_path:
|
||||
print(f"[ERROR] Tool '{tool_name}' not found in {TOOLS_DIR}")
|
||||
print(f"[INFO] Available tools:")
|
||||
for d in TOOLS_DIR.iterdir():
|
||||
if d.is_dir() and not d.name.startswith("_"):
|
||||
print(f" - {d.name}")
|
||||
return False
|
||||
|
||||
# 3. Read local source file
|
||||
if not file_path.exists():
|
||||
print(f"[ERROR] Source file not found: {file_path}")
|
||||
return False
|
||||
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
metadata = _extract_metadata(content)
|
||||
|
||||
if not metadata:
|
||||
print(f"[ERROR] Could not extract metadata from {file_path}")
|
||||
return False
|
||||
|
||||
version = metadata.get("version", "1.0.0")
|
||||
title = metadata.get("title", tool_name)
|
||||
tool_id = metadata.get("id", tool_name).replace("-", "_")
|
||||
|
||||
# 4. Build payload
|
||||
payload = _build_tool_payload(tool_name, file_path, content, metadata)
|
||||
|
||||
# 5. Build headers
|
||||
headers = {
|
||||
"Accept": "application/json",
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
}
|
||||
|
||||
# 6. Send update request through the native tool endpoints
|
||||
update_url = f"{base_url}/api/v1/tools/id/{tool_id}/update"
|
||||
create_url = f"{base_url}/api/v1/tools/create"
|
||||
|
||||
print(f"📦 Deploying tool '{title}' (version {version})...")
|
||||
print(f" File: {file_path}")
|
||||
|
||||
try:
|
||||
# Try update first
|
||||
response = requests.post(
|
||||
update_url,
|
||||
headers=headers,
|
||||
data=json.dumps(payload),
|
||||
timeout=10,
|
||||
)
|
||||
|
||||
if response.status_code == 200:
|
||||
print(f"✅ Successfully updated '{title}' tool!")
|
||||
return True
|
||||
else:
|
||||
print(
|
||||
f"⚠️ Update failed with status {response.status_code}, "
|
||||
"attempting to create instead..."
|
||||
)
|
||||
|
||||
# Try create if update fails
|
||||
res_create = requests.post(
|
||||
create_url,
|
||||
headers=headers,
|
||||
data=json.dumps(payload),
|
||||
timeout=10,
|
||||
)
|
||||
|
||||
if res_create.status_code == 200:
|
||||
print(f"✅ Successfully created '{title}' tool!")
|
||||
return True
|
||||
else:
|
||||
print(f"❌ Failed to update or create. Status: {res_create.status_code}")
|
||||
try:
|
||||
error_msg = res_create.json()
|
||||
print(f" Error: {error_msg}")
|
||||
except:
|
||||
print(f" Response: {res_create.text[:500]}")
|
||||
return False
|
||||
|
||||
except requests.exceptions.ConnectionError:
|
||||
print(
|
||||
"❌ Connection error: Could not reach OpenWebUI at {base_url}"
|
||||
)
|
||||
print(" Make sure OpenWebUI is running and accessible.")
|
||||
return False
|
||||
except requests.exceptions.Timeout:
|
||||
print("❌ Request timeout: OpenWebUI took too long to respond")
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f"❌ Request error: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def list_tools() -> None:
|
||||
"""List all available tools."""
|
||||
print("📋 Available tools:")
|
||||
tools = [d.name for d in TOOLS_DIR.iterdir() if d.is_dir() and not d.name.startswith("_")]
|
||||
|
||||
if not tools:
|
||||
print(" (No tools found)")
|
||||
return
|
||||
|
||||
for tool_name in sorted(tools):
|
||||
tool_dir = TOOLS_DIR / tool_name
|
||||
py_file = _find_tool_file(tool_name)
|
||||
|
||||
if py_file:
|
||||
content = py_file.read_text(encoding="utf-8")
|
||||
metadata = _extract_metadata(content)
|
||||
title = metadata.get("title", tool_name)
|
||||
version = metadata.get("version", "?")
|
||||
print(f" - {tool_name:<30} {title:<40} v{version}")
|
||||
else:
|
||||
print(f" - {tool_name:<30} (no Python file found)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) > 1:
|
||||
if sys.argv[1] == "--list" or sys.argv[1] == "-l":
|
||||
list_tools()
|
||||
else:
|
||||
tool_name = sys.argv[1]
|
||||
success = deploy_tool(tool_name)
|
||||
sys.exit(0 if success else 1)
|
||||
else:
|
||||
# Deploy default tool
|
||||
success = deploy_tool()
|
||||
sys.exit(0 if success else 1)
|
||||
@@ -285,9 +285,8 @@ def format_release_notes(
|
||||
prev_ver = prev_manifest.get("version") or prev.get("version")
|
||||
|
||||
readme_url = _get_readme_url(curr.get("file_path", ""))
|
||||
lines.append(f"- **{curr_title}**: v{prev_ver} → v{curr_ver}")
|
||||
if readme_url:
|
||||
lines.append(f" - 📖 [README]({readme_url})")
|
||||
readme_link = f" | [📖 README]({readme_url})" if readme_url else ""
|
||||
lines.append(f"- **{curr_title}**: v{prev_ver} → v{curr_ver}{readme_link}")
|
||||
lines.append("")
|
||||
|
||||
if comparison["removed"] and not ignore_removed:
|
||||
|
||||
441
scripts/install_all_plugins.py
Normal file
441
scripts/install_all_plugins.py
Normal file
@@ -0,0 +1,441 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Bulk install OpenWebUI plugins from this repository.
|
||||
|
||||
This script installs plugins from the local repository into a target OpenWebUI
|
||||
instance. It only installs plugins that:
|
||||
- live under plugins/actions, plugins/filters, plugins/pipes, or plugins/tools
|
||||
- contain an `openwebui_id` in the plugin header docstring
|
||||
- do not use a Chinese filename
|
||||
- do not use a `_cn.py` localized filename suffix
|
||||
|
||||
Supported Plugin Types:
|
||||
- Action (standard Function class)
|
||||
- Filter (standard Function class)
|
||||
- Pipe (standard Function class)
|
||||
- Tool (native Tools class via /api/v1/tools endpoints)
|
||||
|
||||
Configuration:
|
||||
Create `scripts/.env` with:
|
||||
api_key=sk-your-api-key
|
||||
url=http://localhost:3000
|
||||
|
||||
Usage:
|
||||
python scripts/install_all_plugins.py
|
||||
python scripts/install_all_plugins.py --list
|
||||
python scripts/install_all_plugins.py --dry-run
|
||||
python scripts/install_all_plugins.py --types pipe action filter tool
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Sequence, Tuple
|
||||
|
||||
import requests
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
REPO_ROOT = SCRIPT_DIR.parent
|
||||
ENV_FILE = SCRIPT_DIR / ".env"
|
||||
DEFAULT_TIMEOUT = 20
|
||||
DEFAULT_TYPES = ("pipe", "action", "filter", "tool")
|
||||
SKIP_PREFIXES = ("test_", "verify_")
|
||||
DOCSTRING_PATTERN = re.compile(r'^\s*"""\n(.*?)\n"""', re.DOTALL)
|
||||
|
||||
PLUGIN_TYPE_DIRS = {
|
||||
"action": REPO_ROOT / "plugins" / "actions",
|
||||
"filter": REPO_ROOT / "plugins" / "filters",
|
||||
"pipe": REPO_ROOT / "plugins" / "pipes",
|
||||
"tool": REPO_ROOT / "plugins" / "tools",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PluginCandidate:
|
||||
plugin_type: str
|
||||
file_path: Path
|
||||
metadata: Dict[str, str]
|
||||
content: str
|
||||
function_id: str
|
||||
|
||||
@property
|
||||
def title(self) -> str:
|
||||
return self.metadata.get("title", self.file_path.stem)
|
||||
|
||||
@property
|
||||
def version(self) -> str:
|
||||
return self.metadata.get("version", "unknown")
|
||||
|
||||
|
||||
def _load_env_file(env_path: Path = ENV_FILE) -> Dict[str, str]:
|
||||
values: Dict[str, str] = {}
|
||||
if not env_path.exists():
|
||||
return values
|
||||
|
||||
for raw_line in env_path.read_text(encoding="utf-8").splitlines():
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, value = line.split("=", 1)
|
||||
key_lower = key.strip().lower()
|
||||
values[key_lower] = value.strip().strip('"').strip("'")
|
||||
return values
|
||||
|
||||
|
||||
def load_config(env_path: Path = ENV_FILE) -> Tuple[str, str]:
|
||||
env_values = _load_env_file(env_path)
|
||||
|
||||
api_key = (
|
||||
os.getenv("OPENWEBUI_API_KEY")
|
||||
or os.getenv("api_key")
|
||||
or env_values.get("api_key")
|
||||
or env_values.get("openwebui_api_key")
|
||||
)
|
||||
base_url = (
|
||||
os.getenv("OPENWEBUI_URL")
|
||||
or os.getenv("OPENWEBUI_BASE_URL")
|
||||
or os.getenv("url")
|
||||
or env_values.get("url")
|
||||
or env_values.get("openwebui_url")
|
||||
or env_values.get("openwebui_base_url")
|
||||
)
|
||||
|
||||
missing = []
|
||||
if not api_key:
|
||||
missing.append("api_key")
|
||||
if not base_url:
|
||||
missing.append("url")
|
||||
|
||||
if missing:
|
||||
joined = ", ".join(missing)
|
||||
raise ValueError(
|
||||
f"Missing required config: {joined}. "
|
||||
f"Please set them in environment variables or {env_path}."
|
||||
)
|
||||
|
||||
return api_key, normalize_base_url(base_url)
|
||||
|
||||
|
||||
def normalize_base_url(url: str) -> str:
|
||||
normalized = url.strip()
|
||||
if not normalized:
|
||||
raise ValueError("URL cannot be empty.")
|
||||
return normalized.rstrip("/")
|
||||
|
||||
|
||||
def extract_metadata(content: str) -> Dict[str, str]:
|
||||
match = DOCSTRING_PATTERN.search(content)
|
||||
if not match:
|
||||
return {}
|
||||
|
||||
metadata: Dict[str, str] = {}
|
||||
for raw_line in match.group(1).splitlines():
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith("#") or ":" not in line:
|
||||
continue
|
||||
key, value = line.split(":", 1)
|
||||
metadata[key.strip().lower()] = value.strip()
|
||||
return metadata
|
||||
|
||||
|
||||
def contains_non_ascii_filename(file_path: Path) -> bool:
|
||||
try:
|
||||
file_path.stem.encode("ascii")
|
||||
return False
|
||||
except UnicodeEncodeError:
|
||||
return True
|
||||
|
||||
|
||||
def should_skip_plugin_file(file_path: Path) -> Optional[str]:
|
||||
stem = file_path.stem.lower()
|
||||
|
||||
if contains_non_ascii_filename(file_path):
|
||||
return "non-ascii filename"
|
||||
if stem.endswith("_cn"):
|
||||
return "localized _cn file"
|
||||
if stem.startswith(SKIP_PREFIXES):
|
||||
return "test or helper script"
|
||||
return None
|
||||
|
||||
|
||||
def slugify_function_id(value: str) -> str:
|
||||
slug = re.sub(r"[^a-z0-9]+", "_", value.lower()).strip("_")
|
||||
slug = re.sub(r"_+", "_", slug)
|
||||
return slug or "plugin"
|
||||
|
||||
|
||||
def build_function_id(file_path: Path, metadata: Dict[str, str]) -> str:
|
||||
if metadata.get("id"):
|
||||
return slugify_function_id(metadata["id"])
|
||||
if metadata.get("title"):
|
||||
return slugify_function_id(metadata["title"])
|
||||
return slugify_function_id(file_path.stem)
|
||||
|
||||
|
||||
def has_tools_class(content: str) -> bool:
|
||||
"""Check if plugin content defines a Tools class instead of Function class."""
|
||||
return "\nclass Tools:" in content or "\nclass Tools (" in content
|
||||
|
||||
|
||||
def build_payload(candidate: PluginCandidate) -> Dict[str, object]:
|
||||
manifest = dict(candidate.metadata)
|
||||
manifest.setdefault("title", candidate.title)
|
||||
manifest.setdefault("author", "Fu-Jie")
|
||||
manifest.setdefault(
|
||||
"author_url", "https://github.com/Fu-Jie/openwebui-extensions"
|
||||
)
|
||||
manifest.setdefault("funding_url", "https://github.com/open-webui")
|
||||
manifest.setdefault(
|
||||
"description", f"{candidate.plugin_type.title()} plugin: {candidate.title}"
|
||||
)
|
||||
manifest.setdefault("version", "1.0.0")
|
||||
manifest["type"] = candidate.plugin_type
|
||||
|
||||
if candidate.plugin_type == "tool":
|
||||
return {
|
||||
"id": candidate.function_id,
|
||||
"name": manifest["title"],
|
||||
"meta": {
|
||||
"description": manifest["description"],
|
||||
"manifest": {},
|
||||
},
|
||||
"content": candidate.content,
|
||||
"access_grants": [],
|
||||
}
|
||||
|
||||
return {
|
||||
"id": candidate.function_id,
|
||||
"name": manifest["title"],
|
||||
"meta": {
|
||||
"description": manifest["description"],
|
||||
"manifest": manifest,
|
||||
"type": candidate.plugin_type,
|
||||
},
|
||||
"content": candidate.content,
|
||||
}
|
||||
|
||||
|
||||
def build_api_urls(base_url: str, candidate: PluginCandidate) -> Tuple[str, str]:
|
||||
if candidate.plugin_type == "tool":
|
||||
return (
|
||||
f"{base_url}/api/v1/tools/id/{candidate.function_id}/update",
|
||||
f"{base_url}/api/v1/tools/create",
|
||||
)
|
||||
return (
|
||||
f"{base_url}/api/v1/functions/id/{candidate.function_id}/update",
|
||||
f"{base_url}/api/v1/functions/create",
|
||||
)
|
||||
|
||||
|
||||
def discover_plugins(plugin_types: Sequence[str]) -> Tuple[List[PluginCandidate], List[Tuple[Path, str]]]:
|
||||
candidates: List[PluginCandidate] = []
|
||||
skipped: List[Tuple[Path, str]] = []
|
||||
|
||||
for plugin_type in plugin_types:
|
||||
plugin_dir = PLUGIN_TYPE_DIRS[plugin_type]
|
||||
if not plugin_dir.exists():
|
||||
continue
|
||||
|
||||
for file_path in sorted(plugin_dir.rglob("*.py")):
|
||||
skip_reason = should_skip_plugin_file(file_path)
|
||||
if skip_reason:
|
||||
skipped.append((file_path, skip_reason))
|
||||
continue
|
||||
|
||||
content = file_path.read_text(encoding="utf-8")
|
||||
metadata = extract_metadata(content)
|
||||
if not metadata:
|
||||
skipped.append((file_path, "missing plugin header"))
|
||||
continue
|
||||
if not metadata.get("openwebui_id"):
|
||||
skipped.append((file_path, "missing openwebui_id"))
|
||||
continue
|
||||
|
||||
candidates.append(
|
||||
PluginCandidate(
|
||||
plugin_type=plugin_type,
|
||||
file_path=file_path,
|
||||
metadata=metadata,
|
||||
content=content,
|
||||
function_id=build_function_id(file_path, metadata),
|
||||
)
|
||||
)
|
||||
|
||||
candidates.sort(key=lambda item: (item.plugin_type, item.file_path.as_posix()))
|
||||
skipped.sort(key=lambda item: item[0].as_posix())
|
||||
return candidates, skipped
|
||||
|
||||
|
||||
def install_plugin(
|
||||
candidate: PluginCandidate,
|
||||
api_key: str,
|
||||
base_url: str,
|
||||
timeout: int = DEFAULT_TIMEOUT,
|
||||
) -> Tuple[bool, str]:
|
||||
headers = {
|
||||
"Accept": "application/json",
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
}
|
||||
payload = build_payload(candidate)
|
||||
update_url, create_url = build_api_urls(base_url, candidate)
|
||||
|
||||
try:
|
||||
update_response = requests.post(
|
||||
update_url,
|
||||
headers=headers,
|
||||
data=json.dumps(payload),
|
||||
timeout=timeout,
|
||||
)
|
||||
if 200 <= update_response.status_code < 300:
|
||||
return True, "updated"
|
||||
|
||||
create_response = requests.post(
|
||||
create_url,
|
||||
headers=headers,
|
||||
data=json.dumps(payload),
|
||||
timeout=timeout,
|
||||
)
|
||||
if 200 <= create_response.status_code < 300:
|
||||
return True, "created"
|
||||
|
||||
message = _response_message(create_response)
|
||||
return False, f"create failed ({create_response.status_code}): {message}"
|
||||
except requests.exceptions.Timeout:
|
||||
return False, "request timed out"
|
||||
except requests.exceptions.ConnectionError:
|
||||
return False, f"cannot connect to {base_url}"
|
||||
except Exception as exc:
|
||||
return False, str(exc)
|
||||
|
||||
|
||||
def _response_message(response: requests.Response) -> str:
|
||||
try:
|
||||
return json.dumps(response.json(), ensure_ascii=False)
|
||||
except Exception:
|
||||
return response.text[:500]
|
||||
|
||||
|
||||
def print_candidates(candidates: Sequence[PluginCandidate]) -> None:
|
||||
if not candidates:
|
||||
print("No installable plugins found.")
|
||||
return
|
||||
|
||||
print(f"Found {len(candidates)} installable plugins:")
|
||||
for candidate in candidates:
|
||||
relative_path = candidate.file_path.relative_to(REPO_ROOT)
|
||||
print(
|
||||
f" - [{candidate.plugin_type}] {candidate.title} "
|
||||
f"v{candidate.version} -> {relative_path}"
|
||||
)
|
||||
|
||||
|
||||
def print_skipped_summary(skipped: Sequence[Tuple[Path, str]]) -> None:
|
||||
if not skipped:
|
||||
return
|
||||
|
||||
counts: Dict[str, int] = {}
|
||||
for _, reason in skipped:
|
||||
counts[reason] = counts.get(reason, 0) + 1
|
||||
|
||||
summary = ", ".join(f"{reason}: {count}" for reason, count in sorted(counts.items()))
|
||||
print(f"Skipped {len(skipped)} files ({summary}).")
|
||||
|
||||
|
||||
def parse_args(argv: Optional[Sequence[str]] = None) -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Install repository plugins into an OpenWebUI instance."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--types",
|
||||
nargs="+",
|
||||
choices=sorted(PLUGIN_TYPE_DIRS.keys()),
|
||||
default=list(DEFAULT_TYPES),
|
||||
help="Plugin types to install. Defaults to all supported types.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--list",
|
||||
action="store_true",
|
||||
help="List installable plugins without calling the API.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--dry-run",
|
||||
action="store_true",
|
||||
help="Show what would be installed without calling the API.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--timeout",
|
||||
type=int,
|
||||
default=DEFAULT_TIMEOUT,
|
||||
help=f"Request timeout in seconds. Default: {DEFAULT_TIMEOUT}.",
|
||||
)
|
||||
return parser.parse_args(argv)
|
||||
|
||||
|
||||
def main(argv: Optional[Sequence[str]] = None) -> int:
|
||||
args = parse_args(argv)
|
||||
candidates, skipped = discover_plugins(args.types)
|
||||
|
||||
print_candidates(candidates)
|
||||
print_skipped_summary(skipped)
|
||||
|
||||
if args.list or args.dry_run:
|
||||
return 0
|
||||
|
||||
if not candidates:
|
||||
print("Nothing to install.")
|
||||
return 1
|
||||
|
||||
try:
|
||||
api_key, base_url = load_config()
|
||||
except ValueError as exc:
|
||||
print(f"[ERROR] {exc}")
|
||||
return 1
|
||||
|
||||
print(f"Installing to: {base_url}")
|
||||
|
||||
success_count = 0
|
||||
failed_candidates = []
|
||||
for candidate in candidates:
|
||||
relative_path = candidate.file_path.relative_to(REPO_ROOT)
|
||||
print(
|
||||
f"\nInstalling [{candidate.plugin_type}] {candidate.title} "
|
||||
f"v{candidate.version} ({relative_path})"
|
||||
)
|
||||
ok, message = install_plugin(
|
||||
candidate=candidate,
|
||||
api_key=api_key,
|
||||
base_url=base_url,
|
||||
timeout=args.timeout,
|
||||
)
|
||||
if ok:
|
||||
success_count += 1
|
||||
print(f" [OK] {message}")
|
||||
else:
|
||||
failed_candidates.append(candidate)
|
||||
print(f" [FAILED] {message}")
|
||||
|
||||
print(f"\n" + "="*80)
|
||||
print(
|
||||
f"Finished: {success_count}/{len(candidates)} plugins installed successfully."
|
||||
)
|
||||
|
||||
if failed_candidates:
|
||||
print(f"\n❌ {len(failed_candidates)} plugin(s) failed to install:")
|
||||
for candidate in failed_candidates:
|
||||
print(f" • {candidate.title} ({candidate.plugin_type})")
|
||||
print(f" → Check the error message above")
|
||||
print()
|
||||
|
||||
print("="*80)
|
||||
return 0 if success_count == len(candidates) else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -14,23 +14,23 @@ def main():
|
||||
base_dir = Path(__file__).parent.parent
|
||||
|
||||
print("\n" + "="*80)
|
||||
print("✨ 异步上下文压缩本地部署工具 — 验证状态")
|
||||
print("✨ Async Context Compression Local Deployment Tools — Verification Status")
|
||||
print("="*80 + "\n")
|
||||
|
||||
files_to_check = {
|
||||
"🐍 Python 脚本": [
|
||||
"🐍 Python Scripts": [
|
||||
"scripts/deploy_async_context_compression.py",
|
||||
"scripts/deploy_filter.py",
|
||||
"scripts/deploy_pipe.py",
|
||||
],
|
||||
"📖 部署文档": [
|
||||
"📖 Deployment Documentation": [
|
||||
"scripts/README.md",
|
||||
"scripts/QUICK_START.md",
|
||||
"scripts/DEPLOYMENT_GUIDE.md",
|
||||
"scripts/DEPLOYMENT_SUMMARY.md",
|
||||
"plugins/filters/async-context-compression/DEPLOYMENT_REFERENCE.md",
|
||||
],
|
||||
"🧪 测试文件": [
|
||||
"🧪 Test Files": [
|
||||
"tests/scripts/test_deploy_filter.py",
|
||||
],
|
||||
}
|
||||
@@ -59,24 +59,24 @@ def main():
|
||||
print("\n" + "="*80)
|
||||
|
||||
if all_exist:
|
||||
print("✅ 所有部署工具文件已准备就绪！")
|
||||
print("✅ All deployment tool files are ready!")
|
||||
print("="*80 + "\n")
|
||||
|
||||
print("🚀 快速开始（3 种方式）：\n")
|
||||
print("🚀 Quick Start (3 ways):\n")
|
||||
|
||||
print(" 方式 1: 最简单 (推荐)")
|
||||
print(" Method 1: Easiest (Recommended)")
|
||||
print(" ─────────────────────────────────────────────────────────")
|
||||
print(" cd scripts")
|
||||
print(" python deploy_async_context_compression.py")
|
||||
print()
|
||||
|
||||
print(" 方式 2: 通用工具")
|
||||
print(" Method 2: Generic Tool")
|
||||
print(" ─────────────────────────────────────────────────────────")
|
||||
print(" cd scripts")
|
||||
print(" python deploy_filter.py")
|
||||
print()
|
||||
|
||||
print(" 方式 3: 部署其他 Filter")
|
||||
print(" Method 3: Deploy Other Filters")
|
||||
print(" ─────────────────────────────────────────────────────────")
|
||||
print(" cd scripts")
|
||||
print(" python deploy_filter.py --list")
|
||||
@@ -84,18 +84,18 @@ def main():
|
||||
print()
|
||||
|
||||
print("="*80 + "\n")
|
||||
print("📚 文档参考：\n")
|
||||
print(" • 快速开始: scripts/QUICK_START.md")
|
||||
print(" • 完整指南: scripts/DEPLOYMENT_GUIDE.md")
|
||||
print(" • 技术总结: scripts/DEPLOYMENT_SUMMARY.md")
|
||||
print(" • 脚本说明: scripts/README.md")
|
||||
print(" • 测试覆盖: pytest tests/scripts/test_deploy_filter.py -v")
|
||||
print("📚 Documentation References:\n")
|
||||
print(" • Quick Start: scripts/QUICK_START.md")
|
||||
print(" • Complete Guide: scripts/DEPLOYMENT_GUIDE.md")
|
||||
print(" • Technical Summary: scripts/DEPLOYMENT_SUMMARY.md")
|
||||
print(" • Script Info: scripts/README.md")
|
||||
print(" • Test Coverage: pytest tests/scripts/test_deploy_filter.py -v")
|
||||
print()
|
||||
|
||||
print("="*80 + "\n")
|
||||
return 0
|
||||
else:
|
||||
print("❌ 某些文件缺失！")
|
||||
print("❌ Some files are missing!")
|
||||
print("="*80 + "\n")
|
||||
return 1
|
||||
|
||||
|
||||
173
tests/scripts/test_install_all_plugins.py
Normal file
173
tests/scripts/test_install_all_plugins.py
Normal file
@@ -0,0 +1,173 @@
|
||||
import importlib.util
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
MODULE_PATH = Path(__file__).resolve().parents[2] / "scripts" / "install_all_plugins.py"
|
||||
SPEC = importlib.util.spec_from_file_location("install_all_plugins", MODULE_PATH)
|
||||
install_all_plugins = importlib.util.module_from_spec(SPEC)
|
||||
assert SPEC.loader is not None
|
||||
sys.modules[SPEC.name] = install_all_plugins
|
||||
SPEC.loader.exec_module(install_all_plugins)
|
||||
|
||||
|
||||
PLUGIN_HEADER = '''"""
|
||||
title: Example Plugin
|
||||
version: 1.0.0
|
||||
openwebui_id: 12345678-1234-1234-1234-123456789abc
|
||||
description: Example description.
|
||||
"""
|
||||
'''
|
||||
|
||||
|
||||
def write_plugin(path: Path, header: str) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(header + "\nclass Action:\n pass\n", encoding="utf-8")
|
||||
|
||||
|
||||
def test_should_skip_plugin_file_filters_localized_and_helper_names():
|
||||
assert (
|
||||
install_all_plugins.should_skip_plugin_file(Path("flash_card_cn.py"))
|
||||
== "localized _cn file"
|
||||
)
|
||||
assert (
|
||||
install_all_plugins.should_skip_plugin_file(Path("verify_generation.py"))
|
||||
== "test or helper script"
|
||||
)
|
||||
assert (
|
||||
install_all_plugins.should_skip_plugin_file(Path("测试.py"))
|
||||
== "non-ascii filename"
|
||||
)
|
||||
assert install_all_plugins.should_skip_plugin_file(Path("flash_card.py")) is None
|
||||
|
||||
|
||||
def test_build_function_id_prefers_id_then_title_then_filename():
|
||||
from_id = install_all_plugins.build_function_id(
|
||||
Path("dummy.py"), {"id": "Async Context Compression"}
|
||||
)
|
||||
from_title = install_all_plugins.build_function_id(
|
||||
Path("dummy.py"), {"title": "GitHub Copilot Official SDK Pipe"}
|
||||
)
|
||||
from_file = install_all_plugins.build_function_id(Path("dummy_plugin.py"), {})
|
||||
|
||||
assert from_id == "async_context_compression"
|
||||
assert from_title == "github_copilot_official_sdk_pipe"
|
||||
assert from_file == "dummy_plugin"
|
||||
|
||||
|
||||
def test_build_payload_uses_native_tool_shape_for_tools():
|
||||
candidate = install_all_plugins.PluginCandidate(
|
||||
plugin_type="tool",
|
||||
file_path=Path("plugins/tools/demo/demo_tool.py"),
|
||||
metadata={
|
||||
"title": "Demo Tool",
|
||||
"description": "Demo tool description",
|
||||
"openwebui_id": "12345678-1234-1234-1234-123456789abc",
|
||||
},
|
||||
content='class Tools:\n pass\n',
|
||||
function_id="demo_tool",
|
||||
)
|
||||
|
||||
payload = install_all_plugins.build_payload(candidate)
|
||||
|
||||
assert payload == {
|
||||
"id": "demo_tool",
|
||||
"name": "Demo Tool",
|
||||
"meta": {
|
||||
"description": "Demo tool description",
|
||||
"manifest": {},
|
||||
},
|
||||
"content": 'class Tools:\n pass\n',
|
||||
"access_grants": [],
|
||||
}
|
||||
|
||||
|
||||
def test_build_api_urls_uses_tool_endpoints_for_tools():
|
||||
candidate = install_all_plugins.PluginCandidate(
|
||||
plugin_type="tool",
|
||||
file_path=Path("plugins/tools/demo/demo_tool.py"),
|
||||
metadata={"title": "Demo Tool"},
|
||||
content='class Tools:\n pass\n',
|
||||
function_id="demo_tool",
|
||||
)
|
||||
|
||||
update_url, create_url = install_all_plugins.build_api_urls(
|
||||
"http://localhost:3000", candidate
|
||||
)
|
||||
|
||||
assert update_url == "http://localhost:3000/api/v1/tools/id/demo_tool/update"
|
||||
assert create_url == "http://localhost:3000/api/v1/tools/create"
|
||||
|
||||
|
||||
def test_discover_plugins_only_returns_supported_openwebui_plugins(tmp_path, monkeypatch):
|
||||
actions_dir = tmp_path / "plugins" / "actions"
|
||||
filters_dir = tmp_path / "plugins" / "filters"
|
||||
pipes_dir = tmp_path / "plugins" / "pipes"
|
||||
tools_dir = tmp_path / "plugins" / "tools"
|
||||
|
||||
write_plugin(actions_dir / "flash-card" / "flash_card.py", PLUGIN_HEADER)
|
||||
write_plugin(actions_dir / "flash-card" / "flash_card_cn.py", PLUGIN_HEADER)
|
||||
write_plugin(actions_dir / "infographic" / "verify_generation.py", PLUGIN_HEADER)
|
||||
write_plugin(filters_dir / "missing-id" / "missing_id.py", '"""\ntitle: Missing ID\n"""\n')
|
||||
write_plugin(pipes_dir / "sdk" / "github_copilot_sdk.py", PLUGIN_HEADER)
|
||||
write_plugin(tools_dir / "skills" / "openwebui_skills_manager.py", PLUGIN_HEADER)
|
||||
|
||||
monkeypatch.setattr(
|
||||
install_all_plugins,
|
||||
"PLUGIN_TYPE_DIRS",
|
||||
{
|
||||
"action": actions_dir,
|
||||
"filter": filters_dir,
|
||||
"pipe": pipes_dir,
|
||||
"tool": tools_dir,
|
||||
},
|
||||
)
|
||||
monkeypatch.setattr(install_all_plugins, "REPO_ROOT", tmp_path)
|
||||
|
||||
candidates, skipped = install_all_plugins.discover_plugins(
|
||||
["action", "filter", "pipe", "tool"]
|
||||
)
|
||||
|
||||
candidate_names = [candidate.file_path.name for candidate in candidates]
|
||||
skipped_reasons = {path.name: reason for path, reason in skipped}
|
||||
|
||||
assert candidate_names == [
|
||||
"flash_card.py",
|
||||
"github_copilot_sdk.py",
|
||||
"openwebui_skills_manager.py",
|
||||
]
|
||||
assert skipped_reasons["missing_id.py"] == "missing openwebui_id"
|
||||
assert skipped_reasons["flash_card_cn.py"] == "localized _cn file"
|
||||
assert skipped_reasons["verify_generation.py"] == "test or helper script"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("header", "expected_reason"),
|
||||
[
|
||||
('"""\ntitle: Missing ID\n"""\n', "missing openwebui_id"),
|
||||
("class Action:\n pass\n", "missing plugin header"),
|
||||
],
|
||||
)
|
||||
def test_discover_plugins_reports_missing_metadata(tmp_path, monkeypatch, header, expected_reason):
|
||||
action_dir = tmp_path / "plugins" / "actions"
|
||||
plugin_file = action_dir / "demo" / "demo.py"
|
||||
write_plugin(plugin_file, header)
|
||||
|
||||
monkeypatch.setattr(
|
||||
install_all_plugins,
|
||||
"PLUGIN_TYPE_DIRS",
|
||||
{
|
||||
"action": action_dir,
|
||||
"filter": tmp_path / "plugins" / "filters",
|
||||
"pipe": tmp_path / "plugins" / "pipes",
|
||||
"tool": tmp_path / "plugins" / "tools",
|
||||
},
|
||||
)
|
||||
monkeypatch.setattr(install_all_plugins, "REPO_ROOT", tmp_path)
|
||||
|
||||
candidates, skipped = install_all_plugins.discover_plugins(["action"])
|
||||
|
||||
assert candidates == []
|
||||
assert skipped == [(plugin_file, expected_reason)]
|
||||
Reference in New Issue
Block a user