WPCommunity AI Persona Routing and Automated Configuration Instructions (2026-02-09)
This post is used to document the currently active AI models, Embeddings, and automated triage rules within the community, facilitating unified understanding among forum moderators and contributors.
AI Auto-Tagging has been synchronized to use original_post_only=true (no longer limited to “user’s first post only”)
Embeddings are enabled; current count of public topics missing vectors is 0
5. Rollback and Security
A snapshot and rollback script for this routing adjustment have been generated, supporting one-click restoration.
Additional Reminder: Due to occurrences of plaintext keys during historical debugging, it is advised to continuously implement key rotation and principle of least privilege control.
If you find “inaccurate expert matching” or “inappropriate auto-reply tone” within a specific category, please reply directly to this post with the specific topic link, and we will perform further fine-tuning based on the category.
Next, I will continue to collect an additional 72h of trend data and perform targeted sampling quality checks for the Theme Development/Plugin Development/WooCommerce/Multisite categories.
Embeddings: Enabled on both sites (enabled=true), with selected_model=2 (DashScope embeddings)
3) Translation Pipeline Fix (Critical)
After restoration, it was discovered that the default_llm_id for the translation persona had drifted, causing some translation requests to still route through deepseek-reasoner.
A fix script has been executed in production to align the following personas with ai_translation_model:
Locale detector (ID = -27)
Post translator (ID = -28)
Topic title translator (ID = -29)
Short text translator (ID = -30)
Verification after the fix:
Latest translation call on wpcommunity: llm_id=6 (qwen-plus)
Latest translation call on default: llm_id=4 (qwen-plus)
Note: Historical logs within the recent window may still show older llm_id=2 entries; these reflect pre-fix activity and do not represent current routing behavior.
4) Automation & Observability
wpcommunity automations: 23 total (21 enabled)
24h no-reply reminder (ID = 33)
72h unresolved escalation reminder (ID = 34)
72-hour sampling timer: Running on both default and wpcommunity
Going forward, we will continue tracking the three key metrics — quality, efficiency, and cost — within the 72-hour sampling window.
Each question mandates the following components: problem diagnosis, step-by-step execution instructions, rollback procedure, verification commands, and risk warnings
Queries and gold-standard topic pairs built from high-frequency WordPress questions
Output metrics: Hit@3, Hit@5, Mean Reciprocal Rank (MRR), and zero-result rate
Goal: Ensure “similar-question recommendations before posting” reliably surface actionable answers
3) Next Steps
Run the first baseline round (Bench100 initial batch + full RAG30 set)
Identify and output the Top 10 lowest-scoring questions (Bench100) and Top 10 lowest-hit-rate queries (RAG30)
Feed findings directly back into: persona-based routing logic, knowledge base gap-filling, and automated rule refinement
If you have concrete examples where “AI responses appear correct but are actually non-executable,” please share links in a reply to this post—we’ll incorporate them into our evaluation suite for continuous iteration.
Conclusion: Hit rates remain unchanged; however, HYDE yields a slight degradation in ranking performance (MRR −0.0222) on the current sample set. Thus, HYDE=false remains the default setting.
Next Steps (To Be Executed Immediately)
Complete scoring for the six safety-related questions among the initial batch of 30.
Re-test queries with low ranking (R018, R006, R021, R028) after applying semantic enhancement to their titles and tags.
[Update] Command-Level Guardrails v1.1 Now Implemented
New hard constraints:
• Prohibition of fabricated plugin commands/subcommands;
• Panel/cloud console operations must follow manual step-by-step procedures;
• Command availability must be verified before use.
Retesting of B050/B061/B088: Structures are now more standardized, but plugin-level commands still require stricter validation.
Next steps (v1.2):
• All plugin-level commands must first be validated using wp cli has-command or wp help <command>;
• Where confirmation is impossible, replace with backend procedures + read-only SQL alternatives.
[Update] Command-Level Guardrails v1.2 Deployed and Retested
Enhanced: Plugin-level commands must first be verified for existence (e.g., wp cli has-command COMMAND / wp help COMMAND).
When existence cannot be confirmed, fall back to backend-path access + read-only SQL alternatives; direct command execution is prohibited.
Retested B050/B061/B088: Structure is now more standardized, but complex cases still require localized verification (due to environment differences).
Next step, v1.3: Simultaneously verify both “command existence” and “plugin installation status”; if either fails, only provide backend-path access and read-only SQL solutions.
[Update] Command-Level Guardrails v1.3 Deployed and Retested
Added dual verification: Plugin commands must first verify both “command existence” and “plugin installation status.”
If verification fails, only backend access paths and SQL read-only alternatives are provided; direct command execution is prohibited.
Retested B050/B061/B088: Structure is now more standardized, but for complex questions, a stricter “backend path only (no command) upon failed verification” policy remains necessary.
Next step (v1.4): Upon failed verification, no command execution options will be provided—only backend access paths and SQL read-only solutions.
[Update] Command-Level Guardrails v1.4 Deployed and Retested
New Constraint: When a plugin command fails validation, execution of the command is prohibited; only backend paths and SQL read-only alternatives are permitted.
Retest Results for B050/B061/B088: Structure improved, but plugin commands still occasionally receive execution paths—scoring penalties and template examples require reinforcement.
Next Step (v1.5): Direct point deductions for unvalidated plugin commands, plus additional template example constraints.
