feat(project): sync engineering standards and finalize markdown-normalizer v1.2.7
- Update .github/copilot-instructions.md with latest i18n and naming standards - Add docs/development/issue-reply-guide.md for professional community engagement - Sync all documentation (MKDocs, READMEs, Docs) to v1.2.7 - Include CI/CD and Agent instruction templates for better automation
This commit is contained in:
fujie
@@ -4,15 +4,17 @@
|
||||
|
||||
## 📚 Table of Contents
|
||||
|
||||
1. [Quick Start](#1-quick-start)
|
||||
2. [Core Concepts & SDK Details](#2-core-concepts--sdk-details)
|
||||
3. [Deep Dive into Plugin Types](#3-deep-dive-into-plugin-types)
|
||||
* [Action](#31-action)
|
||||
* [Filter](#32-filter)
|
||||
* [Pipe](#33-pipe)
|
||||
4. [Advanced Development Patterns](#4-advanced-development-patterns)
|
||||
5. [Best Practices & Design Principles](#5-best-practices--design-principles)
|
||||
6. [Troubleshooting](#6-troubleshooting)
|
||||
1. [Quick Start](#1-quick-start)
|
||||
2. [Core Concepts & SDK Details](#2-core-concepts--sdk-details)
|
||||
3. [Deep Dive into Plugin Types](#3-deep-dive-into-plugin-types)
|
||||
* [Action](#31-action)
|
||||
* [Filter](#32-filter)
|
||||
* [Pipe](#33-pipe)
|
||||
4. [Advanced Development Patterns](#4-advanced-development-patterns)
|
||||
5. [Best Practices & Design Principles](#5-best-practices--design-principles)
|
||||
6. [Repository Standards (openwebui-extensions)](#6-repository-standards-openwebui-extensions)
|
||||
7. [Custom Agent Design Recommendations](#7-custom-agent-design-recommendations)
|
||||
8. [Troubleshooting](#8-troubleshooting)
|
||||
|
||||
---
|
||||
|
||||
@@ -21,9 +23,10 @@
|
||||
### 1.1 What are OpenWebUI Plugins?
|
||||
|
||||
OpenWebUI Plugins (officially called "Functions") are the primary way to extend the platform's capabilities. Running in a backend Python environment, they allow you to:
|
||||
* 🔌 **Integrate New Models**: Connect to Claude, Gemini, or custom RAGs via Pipes.
|
||||
* 🎨 **Enhance Interaction**: Add buttons (e.g., "Export", "Generate Chart") next to messages via Actions.
|
||||
* 🔧 **Intervene in Processes**: Modify data before requests or after responses (e.g., inject context, filter sensitive words) via Filters.
|
||||
|
||||
* 🔌 **Integrate New Models**: Connect to Claude, Gemini, or custom RAGs via Pipes.
|
||||
* 🎨 **Enhance Interaction**: Add buttons (e.g., "Export", "Generate Chart") next to messages via Actions.
|
||||
* 🔧 **Intervene in Processes**: modify data before requests or after responses (e.g., inject context, filter sensitive words) via Filters.
|
||||
|
||||
### 1.2 Your First Plugin (Hello World)
|
||||
|
||||
@@ -69,9 +72,10 @@ class Action:
|
||||
### 2.1 ⚠️ Important: Sync vs Async
|
||||
|
||||
OpenWebUI plugins run within an `asyncio` event loop.
|
||||
* **Principle**: All I/O operations (database, file, network) must be non-blocking.
|
||||
* **Pitfall**: Calling synchronous methods directly (e.g., `time.sleep`, `requests.get`) will freeze the entire server.
|
||||
* **Solution**: Wrap synchronous calls using `await asyncio.to_thread(sync_func, ...)`.
|
||||
|
||||
* **Principle**: All I/O operations (database, file, network) must be non-blocking.
|
||||
* **Pitfall**: Calling synchronous methods directly (e.g., `time.sleep`, `requests.get`) will freeze the entire server.
|
||||
* **Solution**: Wrap synchronous calls using `await asyncio.to_thread(sync_func, ...)`.
|
||||
|
||||
### 2.2 Core Parameters
|
||||
|
||||
@@ -88,8 +92,8 @@ All plugin methods (`inlet`, `outlet`, `pipe`, `action`) support injecting the f
|
||||
|
||||
### 2.3 Configuration System (Valves)
|
||||
|
||||
* **`Valves`**: Global admin configuration.
|
||||
* **`UserValves`**: User-level configuration (higher priority, overrides global).
|
||||
* **`Valves`**: Global admin configuration.
|
||||
* **`UserValves`**: User-level configuration (higher priority, overrides global).
|
||||
|
||||
```python
|
||||
class Filter:
|
||||
@@ -113,7 +117,7 @@ class Filter:
|
||||
|
||||
**Role**: Adds buttons below messages that trigger upon user click.
|
||||
|
||||
**Advanced Usage: Execute JavaScript on Frontend (File Download Example)**
|
||||
#### Advanced Usage: Execute JavaScript on Frontend (File Download Example)
|
||||
|
||||
```python
|
||||
import base64
|
||||
@@ -138,11 +142,11 @@ async def action(self, body, __event_call__):
|
||||
|
||||
**Role**: Middleware that intercepts and modifies requests/responses.
|
||||
|
||||
* **`inlet`**: Before request. Used for injecting context, modifying model parameters.
|
||||
* **`outlet`**: After response. Used for formatting output, logging.
|
||||
* **`stream`**: During streaming. Used for real-time sensitive word filtering.
|
||||
* **`inlet`**: Before request. Used for injecting context, modifying model parameters.
|
||||
* **`outlet`**: After response. Used for formatting output, logging.
|
||||
* **`stream`**: During streaming. Used for real-time sensitive word filtering.
|
||||
|
||||
**Example: Injecting Environment Variables**
|
||||
#### Example: Injecting Environment Variables
|
||||
|
||||
```python
|
||||
async def inlet(self, body, __metadata__):
|
||||
@@ -159,7 +163,7 @@ async def inlet(self, body, __metadata__):
|
||||
|
||||
**Role**: Custom Model/Agent.
|
||||
|
||||
**Example: Simple OpenAI Wrapper**
|
||||
#### Example: Simple OpenAI Wrapper
|
||||
|
||||
```python
|
||||
import requests
|
||||
@@ -180,11 +184,14 @@ class Pipe:
|
||||
## 4. Advanced Development Patterns
|
||||
|
||||
### 4.1 Pipe & Filter Collaboration
|
||||
|
||||
Use `__request__.app.state` to share data between plugins.
|
||||
* **Pipe**: `__request__.app.state.search_results = [...]`
|
||||
* **Filter (Outlet)**: Read `search_results` and format them as citation links appended to the response.
|
||||
|
||||
* **Pipe**: `__request__.app.state.search_results = [...]`
|
||||
* **Filter (Outlet)**: Read `search_results` and format them as citation links appended to the response.
|
||||
|
||||
### 4.2 Async Background Tasks
|
||||
|
||||
Execute time-consuming operations (e.g., summarization, database storage) in the background without blocking the user response.
|
||||
|
||||
```python
|
||||
@@ -204,15 +211,18 @@ async def background_job(self, chat_id):
|
||||
## 5. Best Practices & Design Principles
|
||||
|
||||
### 5.1 Naming & Positioning
|
||||
* **Short & Punchy**: e.g., "FlashCard", "DeepRead". Avoid generic terms like "Text Analysis Assistant".
|
||||
* **Complementary**: Don't reinvent the wheel; clarify what specific problem your plugin solves.
|
||||
|
||||
* **Short & Punchy**: e.g., "FlashCard", "DeepRead". Avoid generic terms like "Text Analysis Assistant".
|
||||
* **Complementary**: Don't reinvent the wheel; clarify what specific problem your plugin solves.
|
||||
|
||||
### 5.2 User Experience (UX)
|
||||
* **Timely Feedback**: Send a `notification` ("Generating...") before time-consuming operations.
|
||||
* **Visual Appeal**: When Action outputs HTML, use modern CSS (rounded corners, shadows, gradients).
|
||||
* **Smart Guidance**: If text is too short, prompt the user: "Suggest entering more content for better results".
|
||||
|
||||
* **Timely Feedback**: Send a `notification` ("Generating...") before time-consuming operations.
|
||||
* **Visual Appeal**: When Action outputs HTML, use modern CSS (rounded corners, shadows, gradients).
|
||||
* **Smart Guidance**: If text is too short, prompt the user: "Suggest entering more content for better results".
|
||||
|
||||
### 5.3 Error Handling
|
||||
|
||||
Never let a plugin fail silently. Catch exceptions and inform the user via `__event_emitter__`.
|
||||
|
||||
```python
|
||||
@@ -227,8 +237,127 @@ except Exception as e:
|
||||
|
||||
---
|
||||
|
||||
## 6. Troubleshooting
|
||||
## 6. Repository Standards (openwebui-extensions)
|
||||
|
||||
* **HTML not showing?** Ensure it's wrapped in a ` ```html ... ``` ` code block.
|
||||
* **Database error?** Check if you called synchronous DB methods directly in an `async` function; use `asyncio.to_thread`.
|
||||
* **Parameters not working?** Check if `Valves` are defined correctly and if they are being overridden by `UserValves`.
|
||||
### 6.1 Single-file i18n Requirement
|
||||
|
||||
In this repository, each plugin must use a **single source file** with built-in i18n logic. Do not split source code by language.
|
||||
|
||||
* Required pattern: `plugins/{type}/{name}/{name}.py`
|
||||
* Required docs: `README.md` + `README_CN.md`
|
||||
|
||||
### 6.2 Safe Context Access (Required)
|
||||
|
||||
Prefer helper methods like `_get_user_context` and `_get_chat_context` instead of direct, fragile field access from `__user__` / `body`.
|
||||
|
||||
### 6.3 Event and Logging Conventions
|
||||
|
||||
* Use status/notification events for user-visible progress.
|
||||
* Use frontend console debug logs (`execute`) for live debugging during development.
|
||||
* Use Python `logging` for backend logs; avoid `print()` in production plugin code.
|
||||
|
||||
### 6.4 Frontend Language Detection and Timeout Guard
|
||||
|
||||
When reading frontend language via `__event_call__`, always use:
|
||||
|
||||
* JS `try...catch` fallback return
|
||||
* backend `asyncio.wait_for(..., timeout=2.0)`
|
||||
|
||||
This prevents deadlocks when frontend execution fails.
|
||||
|
||||
### 6.5 Copilot SDK Tool Definition
|
||||
|
||||
For custom Copilot SDK tools, define explicit parameter schema using a `pydantic.BaseModel` and pass it with `params_type` in `define_tool(...)`.
|
||||
|
||||
### 6.6 Copilot SDK Streaming Output Format
|
||||
|
||||
* Use native `<think>...</think>` for reasoning output.
|
||||
* Ensure `</think>` is closed before normal content or tool cards.
|
||||
* For tool result cards, use native `<details type="tool_calls" ...>` format.
|
||||
* Escape attribute quotes in `arguments` and `result` as `"`.
|
||||
|
||||
### 6.7 Source-derived Production Patterns (Recommended)
|
||||
|
||||
The following patterns are extracted from `github_copilot_sdk.py` and `workspace_file_manager.py`:
|
||||
|
||||
* **Tool parameter anti-drift**: define tools with `params_type=BaseModel`, and execute with `model_dump(exclude_unset=True)` so missing params do not become explicit `None`.
|
||||
* **Tool name normalization**: enforce `^[a-zA-Z0-9_-]+$`; if non-ASCII names collapse, use an `md5` suffix fallback to keep registration stable.
|
||||
* **Workspace sandboxing**: resolve and verify every path stays inside the workspace root to prevent traversal.
|
||||
* **3-step file delivery**: local write -> `publish_file_from_workspace` -> return `/api/v1/files/{id}/content`, with `skip_rag=true` metadata.
|
||||
* **Dual upload channel**: prefer API upload (S3-compatible), fallback to DB + local copy.
|
||||
* **Streaming stability**: close `<think>` before emitting `assistant.message_delta` content.
|
||||
* **Native tool cards**: emit `<details type="tool_calls">` on `tool.execution_complete` with strict HTML escaping (`"`, newline escaping).
|
||||
* **TODO persistence linkage**: on successful `update_todo`, sync both `TODO.md` and database state.
|
||||
|
||||
### 6.8 Full Source-derived Knowledge Base (from `plugins/`)
|
||||
|
||||
The following is a broader extraction from `actions/`, `filters/`, `pipes/`, `pipelines/`, and `tools/`:
|
||||
|
||||
* **Action input hygiene**: normalize multimodal message content, strip old plugin HTML blocks (`OPENWEBUI_PLUGIN_OUTPUT`), and enforce minimum text length before expensive model calls.
|
||||
* **Action i18n hardening**: use `TRANSLATIONS + fallback_map + base-lang fallback` (`fr-CA -> fr-FR`, `en-GB -> en-US`), keep all status/UI/JS strings in i18n keys, and protect `format(**kwargs)` formatting.
|
||||
* **Frontend language detection (production-safe)**: use priority chain `document.lang -> localStorage(locale/language) -> navigator.language -> profile/request`, and always wrap `__event_call__(execute)` with timeout.
|
||||
* **Long-running UX pattern**: emit immediate `status + notification`, report staged progress (`analyzing/rendering/saving`), and keep detailed exception data in backend logs.
|
||||
* **HTML plugin composability**: use insertion markers for style/content/script, support both overwrite (`CLEAR_PREVIOUS_HTML`) and merge mode, and keep wrappers deterministic.
|
||||
* **Theme-aware iframe rendering**: detect theme from parent meta/class/data-theme with system fallback, and inject theme-aware colors for SVG/PNG export.
|
||||
* **Client-side render-and-export pipeline**: render offscreen chart/mindmap, export SVG/PNG, upload via `/api/v1/files/`, and persist updates through event API + chat persistence API.
|
||||
* **DOCX export production patterns**: apply `TITLE_SOURCE` fallback chain (`chat_title -> markdown_title -> user+date`), remove reasoning blocks, convert LaTeX via `latex2mathml + mathml2omml`, and emit citation-aware references/bookmarks.
|
||||
* **OpenWebUI file retrieval fallback ladder**: DB inline bytes/base64 -> S3 direct read -> local path variants -> public URL -> internal `/api/v1/files/{id}/content` -> raw object attrs, with max-byte guards at every stage.
|
||||
* **Filter singleton-safe design**: never store request-scoped mutable state on `self`; compute per-request values from `body` and context helpers.
|
||||
* **Async context compression patterns**: two-phase flow (`inlet` apply summary, `outlet` async generate summary), model-level threshold overrides, fast estimate + precise count near limit, and system-message protection (`effective_keep_first`).
|
||||
* **Model compatibility guardrails**: skip incompatible model families (e.g., `copilot_sdk` paths) and avoid hardcoded default model IDs.
|
||||
* **Folder memory pattern**: trigger periodic rule extraction (`every N messages`), replace rules idempotently using block markers (`RULES_BLOCK_START/END`), and optionally update root folder.
|
||||
* **Tool workspace hardening**: all file APIs (`list/read/write/delete/publish`) must re-check sandbox boundary, enforce size limits, and return user-ready download hints.
|
||||
* **MoE prompt refiner pattern (pipeline)**: detect aggregation prompts via trigger prefix, parse original query + segmented responses, then rewrite to synthesis-oriented master prompt with optional aggregation model reroute.
|
||||
|
||||
### 6.9 Copilot-related Engineering Configuration
|
||||
|
||||
To support plugin engineering with **GitHub Copilot + Gemini CLI + antigravity mode**, adopt these controls:
|
||||
|
||||
* **Primary/secondary assistant lanes**: Copilot is primary implementation lane; Gemini CLI is secondary draft/verification lane.
|
||||
* **Single merge contract**: both lanes must pass the same repository constraints (single-file i18n, context helpers, event conventions, release workflow rules).
|
||||
* **Tool schema discipline**: all Copilot SDK tools use explicit `params_type` with Pydantic models.
|
||||
* **Antigravity safety**: small reversible edits, timeout guards, fallback routing, and deterministic file/output paths.
|
||||
* **File creation protocol**: write in workspace scope, publish via workspace publish flow, return `/api/v1/files/{id}/content` for delivery.
|
||||
|
||||
Detailed design document:
|
||||
|
||||
* `docs/development/copilot-engineering-plan.md`
|
||||
|
||||
---
|
||||
|
||||
## 7. Custom Agent Design Recommendations
|
||||
|
||||
### 7.1 Suggested architecture (for this repo)
|
||||
|
||||
* **Orchestrator Pipe**: session lifecycle, model routing, streaming events.
|
||||
* **Tool Adapter Layer**: unify OpenWebUI Tools / OpenAPI / MCP with param validation and name normalization.
|
||||
* **Workspace I/O Layer**: sandboxed file operations + publish pipeline.
|
||||
* **Render Layer**: `<think>` lifecycle, tool cards, status/notification events.
|
||||
|
||||
### 7.2 MVP checklist
|
||||
|
||||
1. Dual config model: `Valves + UserValves` (user overrides first).
|
||||
2. Unified context helpers: `_get_user_context` / `_get_chat_context`.
|
||||
3. At least one artifact-delivery tool (e.g., `publish_file_from_workspace`).
|
||||
4. Minimal streaming loop: `reasoning_delta`, `message_delta`, `tool.execution_complete`.
|
||||
5. Unified error reporting via notification events.
|
||||
|
||||
### 7.3 Three high-impact agents you can build now
|
||||
|
||||
* **Repo Analyst Agent**: output architecture map, risk list, and refactor proposals.
|
||||
* **Release Draft Agent**: generate Conventional Commit title/body + bilingual release summary.
|
||||
* **Docs Sync Agent**: compare source/doc versions and output a concrete sync file list.
|
||||
|
||||
### 7.4 Implementation priority
|
||||
|
||||
* **P0**: Release Draft Agent (highest ROI, lowest risk).
|
||||
* **P1**: Docs Sync Agent (reduces doc drift).
|
||||
* **P2**: Repo Analyst Agent (medium/long-term evolution).
|
||||
|
||||
---
|
||||
|
||||
## 8. Troubleshooting
|
||||
|
||||
* **HTML not showing?** Ensure it's wrapped in a ` ```html ... ``` ` code block.
|
||||
* **Database error?** Check if you called synchronous DB methods directly in an `async` function; use `asyncio.to_thread`.
|
||||
* **Parameters not working?** Check if `Valves` are defined correctly and if they are being overridden by `UserValves`.
|
||||
|
||||
Reference in New Issue
Block a user