Getting it into your agent
One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.
npx skills add THU-SAGE/syll --skill gui-agentgit clone --depth 1 https://github.com/THU-SAGE/syllWrote this? Show the measurements
A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.
[](https://agentmods.dev/skills/thu-sage/syll/gui-agent)<a href="https://agentmods.dev/skills/thu-sage/syll/gui-agent"><img src="https://agentmods.dev/badge/skills/thu-sage/syll/gui-agent.svg" alt="Measured on agentmods" height="20"></a>What it costs to keep this loaded
Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.
| Model | Per session | Once invoked |
|---|---|---|
| Fable 5.1 | $0.00013 | $0.00697 |
| Opus 5 | $0.00006 | $0.00349 |
| Sonnet 5 | $0.00003 | $0.00139 |
| Haiku 4.5 | $0.00001 | $0.00070 |
Grade A, and why
gui-agent scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured 7d ago.
A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.
Nothing flagged
None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.
How it starts
The opening of the file, as written. The whole thing — 86 lines — stays where its author put it; the contents beside it link to each section on GitHub.
GUI Agent
You can interact with desktop applications using the gui_action tool.
How It Works
- The tool captures a screenshot of the current screen
- Sends it to the UI-TARS vision model for analysis
- UI-TARS returns a thought + action (click, type, scroll, etc.)
- The action is executed via pyautogui
- Steps repeat until the task is complete or max steps reached
Usage Protocol
When the user asks you to perform a GUI task:
- Understand the goal - Break the task into clear steps
- Call
gui_actionwith a clear instruction - Review the result - Check the returned screenshot and status
- Iterate if needed - Call again with refined instructions
Example
User: "Open Chrome and search for 'Syll'"
You should call:
gui_action(instruction="Open Chrome browser, click on the address bar, type 'Syll' and press Enter")
Safety Rules
-
Never perform destructive actions without user confirmation:
- Deleting files or folders
- Closing unsaved documents
- System settings changes
- Financial transactions
- Sending messages/emails
-
Always verify the current screen state before acting
-
Stop immediately if the screen shows unexpected content (login pages with sensitive data, etc.)
-
Report clearly what actions were taken and their results
Supported Actions
| Action | Description | Example |
|---|---|---|
click |
Left click at position | click(point='<point>500 300</point>') |
right_click |
Right click | right_click(point='<point>500 300</point>') |
double_click |
Double click | double_click(point='<point>500 300</point>') |
drag |
Drag from A to B | drag(start='<point>100 100</point>', end='<point>200 200</point>') |
type |
Type text | type(content='hello world') |
hotkey |
Key combination | hotkey(key='ctrl+c') |
scroll |
Scroll up/down | scroll(point='<point>500 300</point>', direction='down', amount=3) |
wait |
Pause | wait(seconds=2) |
finished |
Task complete | finished(content='Done') |
What this file has done since we first saw it
Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.
- 7d ago First seen · 86 lines · 13 tokens per session scan A 1e223833066a
gui-agent is a skill published in the GitHub repository THU-SAGE/syll (303 stars, last pushed 3mo ago), licensed MIT. It adds 13 tokens to every session and 697 once invoked, about $0.0001 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-08-30.
Other skills, from other repositories
opencli-sitemap-author
Use when creating or maintaining OpenCLI site sitemaps: agent-facing navigation, page-state, action, workflow, API-reference, pitfall, and fallback knowledge for a website. Use after browser exploration discovers durable site context, when a sitemap is stale, or when promoting local site knowledge into the repo.
debug-optimize-lcp
Guides debugging and optimizing Largest Contentful Paint (LCP) using Chrome DevTools MCP tools. Use this skill whenever the user asks about LCP performance, slow page loads, Core Web Vitals optimization, or wants to understand why their page's main content takes too long to appear. Also use when the user mentions…
interactive-login
How to complete browser/interactive logins (aws / gh / glab / gcloud). The platform backgrounds the login poller so it survives the human's browser round-trip — and when that does NOT work.
pinchtab-mcp
Use this skill when a task requires browser automation through PinchTab's MCP server connected to a remote browser instance. Covers navigation, element interaction, data extraction, form filling, multi-step flows, and session management via MCP tools.
azure-messaging-webpubsub-java
Build real-time web applications with Azure Web PubSub SDK for Java. Use when implementing WebSocket-based messaging, live updates, chat applications, or server-to-client push notifications.
google-safe-browsing
Prevent and fix Google Safe Browsing "Dangerous site" flags. Use when launching a public web app, buying/picking a domain, building a login or signup page, or when any site shows a red "Dangerous site" / "Deceptive site" warning in Chrome, Brave, Safari, Firefox, or Edge. Triggers on "dangerous site", "deceptive…