smart-data-collection

smart-data-collection is a skill for Claude Code, Codex from UnicomAI/wanwu. It costs 115 tokens per session (3,612 once invoked), scanned A, original, Apache-2.0.

A workflow for extracting structured information from images and documents such as PDFs, Word files, and spreadsheets, then storing it in a database.

In plain words
What is it for?
Use it for batch document extraction, image-to-data work, field mapping, SQL generation, and controlled data imports.
Why use it?
It reduces manual copying from unstructured files and keeps extracted fields mapped consistently before import.

Skill for Claude CodeCodex

About the project

Wanwu is an enterprise platform for building AI agents, workflows, retrieval-augmented applications, and managing models in multi-tenant environments. It is designed for developers and enterprise teams delivering AI applications and integrations. The catalogue entries provide skills and agents for using the platform.

UnicomAI/wanwu · 2,456 stars · on GitHub

Install

Getting it into your agent

One page per mod, every tool's command on it. A separate URL per tool would split the same page into five that compete with each other.

agentmods
npx agentmods add skills/unicomai/wanwu/smart-data-collection
Any agent
npx skills add UnicomAI/wanwu --skill smart-data-collection
Clone the repo
git clone --depth 1 https://github.com/UnicomAI/wanwu

Made for: Claude Code, Codex.

Wrote this? Show the measurements

A badge with what this costs and how it scanned, read live from this page, so it follows the numbers instead of freezing them. Markdown for a README, HTML for a documentation site or a project page.

agentmods badge for smart-data-collection

README.md
[![agentmods](https://agentmods.dev/badge/skills/unicomai/wanwu/smart-data-collection.svg)](https://agentmods.dev/skills/unicomai/wanwu/smart-data-collection)
Your own site
<a href="https://agentmods.dev/skills/unicomai/wanwu/smart-data-collection"><img src="https://agentmods.dev/badge/skills/unicomai/wanwu/smart-data-collection.svg" alt="Measured on agentmods" height="20"></a>
Per session 115 Skills are progressive disclosure: only the name and description are preloaded; the body loads when the skill is used.
When invoked 3,612 The whole file, excluding the scripts and references it only reads on demand.
Security scan A 0 findings. Scan, not verified.
Origin original No closer match found in the catalogue.
Token cost

What it costs to keep this loaded

Counted locally with the o200k_base tokenizer, which is exact for GPT models; Claude uses its own tokenizer and its counts differ. Treat this as one consistent yardstick across the catalogue rather than a bill. Prices are per million input tokens.

ModelPer sessionOnce invoked
Fable 5 $0.00115 $0.03612
Opus 5 $0.00057 $0.01806
Sonnet 5 $0.00023 $0.00722
Haiku 4.5 $0.00012 $0.00361

Measured yesterday against content hash 4e9838810c45, method: parsed. Prices are Anthropic first-party input rates as of 2026-08-30, from the pricing page.

Security

Grade A, and why

smart-data-collection scanned grade A with 0 findings against 26 rules in 11 categories — prompt injection, anti-refusal, data exfiltration, privilege escalation, supply chain, agent snooping, system-prompt leakage, SSRF and excessive agency — measured yesterday.

A static scan of the body, not an audit. Every finding is printed with the line that produced it so you can judge whether it matters here. A mod is markdown that instructs an agent; that is exactly why what it instructs is worth reading.

Nothing flagged

None of the 26 patterns this scan looks for appear in this file: no shell pipes, no recursive deletes, no credential paths, no hidden text, no instruction-override or anti-refusal phrasing, no agent-config snooping. That is not a guarantee, it is the absence of the things that are checkable.

configs/microservice/bff-service/configs/agent-skills/ontology/smart-data-collection/SKILL.md · 162 lines

How it starts

The opening of the file, as written. The whole thing — 162 lines — stays where its author put it; the contents beside it link to each section on GitHub.

Smart Data Collection(数据采集 / 写入)

本 skill 定义 固定顺序 的数据写入工具链,完全由 ontology native 子命令bkn / dataview / ds)实现; 所有 CLI 执行均委托 ontology-core

安全红线(IRON RULE)

  1. 严禁直连数据库:不得使用 mysql / mysqldump / pymysql / sqlalchemy+pymysql / 直接 JDBC 连接等方式向 data_source 写入数据。
    • ontology ds get 返回的 bin_data.password平台加密串,不是明文密码;尝试解密或直连均视为违规。
    • 直连绕过平台审计、权限、字段映射、索引刷新与事务一致性,会导致 BKN 索引与底表脱节。
  2. 唯一允许的写入入口ontology ds import-csv <datasource_id> --table-name <table> --file <csv> [...]
    • 由 CLI 调用后端 POST /api/automation/v1/data-flow/flow,由平台 data-flow 服务统一执行写入与回执。
  3. import-csv 仅支持 INSERT,不支持 UPDATE/UPSERT/REPLACE
    • 每行执行一次 INSERT;若主键已存在,MySQL 直接报 Error 1062 (23000): Duplicate entry '<pk>' for key '<table>.PRIMARY',整批失败、summary.failed=1rows_written=0
    • 导入前必须先查"哪些主键已存在",把这些行从 CSV 剔除;不要假设 data-flow 会自动覆盖或合并。
    • 修改/删除已有行:本通道不支持,直接拒绝并告知用户。
  4. 写操作需明确确认:导入前必须先复述本次将写入的 kn / object_type / dataview / datasource / table_name / row_count,得到用户确认后再执行。
  5. 不得用 dataview query --sql / --raw-sql 执行 INSERT/UPDATE/DELETE:该入口在本部署只允许 SELECT/WITH,不是写入通道。

调用方式(统一 ontology 命令;委托 ontology-core 执行)

smart-data-collection(本 skill:定义写入工具链与顺序)
  └─ ontology-core(实际执行 ontology 命令)
ontology --user-id <accountId> <command> [options]
  • --user-id <accountId>:必传(顶层选项,写在子命令之前)。
  • 网关 / ONTOLOGY_BASE_URL 由 ontology-core 承担。
  • 本部署 CLI 无 token;本 skill 不出现 --token / Authorization
  • -bd bd_public:默认值,可省。

Phase 0:非 CSV 输入的前置解析(图片 / PDF / Word / Excel)

当用户给的不是结构化 CSV,而是 .jpg / .jpeg / .png / .bmp / .pdf / .doc / .docx / .xls / .xlsx,必须先经 Phase 0 转成 CSV,再进入下面的第 1~10 步主流程。Phase 0 由本 skill 编排,绝不新增「直接生成 INSERT SQL」或「直连 mysql」通道,IRON RULE 全部保留。

  • 默认 KNe0e2ed66-8eff-4f14-8e15-9fd6171f8e53;用户指定时优先。参数名统一 kn_id
  • Phase 0 子步骤
    • 0a. 文件 → Markdown
      • 图片:通过 Skill 工具显式调用 yj-ocr-parser skill(契约见 skills/yj-ocr-parser/SKILL.md),输出 Markdown(含表头、表格行)。
      • 文档(PDF / Word / Excel):调用文档解析工具(具体工具待集成),输出统一 Markdown 表 + 字段样值。
      • 完整契约见 references/parse-input.md
    • 0b. 字段语义映射 + 用户逐条确认
      • 委托 ontology-core 调 bkn object-type query/getdata_properties[](字段名 / 类型 / required / enum),作为映射目标的唯一字段定义来源
      • 名称不一致 / 类型不一致 / 必填缺失 / 歧义 任一触发 → 必须停下来给用户出对照表,再一次只就一条差异项单独确认,拿到该条决策才问下一条;严禁把多条揉成一个 是否确认以上映射? 的总确认。未确认前禁止写出 CSV。
      • 完整契约见 references/map-fields.md
    • 0c. 写出 CSV
      • 按用户确认后的映射结果落成 UTF-8 + RFC 4180 的 CSV;列名 = 映射后字段名(与 dataview.fields[].name 对齐,便于第 4 步无歧义消费)。
      • 输出文件路径作为后续第 4 步 CSV 准备的输入。
  • Phase 0 边界
    • 不生成任何 .sql 文件、不生成 INSERT/UPDATE/DELETE 语句、不连 mysql、不读 bin_data.password
    • 用户给的就是 CSV 时,跳过整个 Phase 0,直接从下面第 1 步开始。

Read the full file on GitHub · 162 lines

Files

What ships with it

5 files beside SKILL.md in the same directory: the scripts, references and assets a skill reads on demand. Not counted in the per-session cost; read them before you install if any of them is executable.

Changes

What this file has done since we first saw it

Hashed on every crawl. A supply-chain change to an agent config is a question of when, not whether, so the history is kept rather than the latest state alone.

  1. yesterday First seen · 162 lines · 115 tokens per session scan A 4e9838810c45

Subscribe to this mod's changes

smart-data-collection is a skill published in the GitHub repository UnicomAI/wanwu (2,456 stars, last pushed today), licensed Apache-2.0. It adds 115 tokens to every session and 3,612 once invoked, about $0.0006 per session on Opus 5. A static security scan graded it A with 0 findings. No closer match exists in the catalogue, so it is treated as the original; first seen 2026-09-03.

Related

Other skills, from other repositories

meta-multi-format-export-pack

From one piece of source content, render four deliverables: .docx report, .pptx slides, .xlsx data, and an HTML/PDF public version.

opensquilla/opensquilla · 41 tokens

superlinked-docs

Offload document, image, and structured-output work to the Superlinked inference cluster: convert PDF/DOCX/PPTX/XLSX/HTML/scans to clean markdown, describe an image (caption + tags), or produce schema/grammar-constrained JSON off the cluster — instead of ingesting the file directly, which can reduce the tokens billed…

superlinked/sie · 125 tokens

resilient-context-extraction

Ensures agents extract data from context files with validation and fallback strategies before resorting to assumptions or external searches.

HKUDS/OpenSpace · 29 tokens

markdown-converter

Convert binary documents (PDF, DOCX, XLSX, PPTX, HTML, EPUB, images) to clean LLM-friendly Markdown using Microsoft's markitdown Python tool. Use when a user attaches a binary file and you need to read its contents.

Team-Commonly/commonly · 57 tokens

prioritize-context-data

Ensures agents check and use provided context files for data before attempting external searches.

HKUDS/OpenSpace · 21 tokens

lark-base

【何时用:仅当用户明确指向飞书/Lark(发到飞书、飞书文档等)时使用;泛指做个文档或PPT或表格或方案默认走本地工具,不要误用飞书】飞书多维表格(Base)操作:建表、字段、记录、视图、统计、公式/lookup、表单、仪表盘、应用模式(BaseApp/AppMode 页面与组件)、Workspace 目录、workflow、角色权限;遇到 Base/多维表格/bitable、BaseApp/AppMode 或 /app/ 链接时使用。BaseApp 不走 lark-apps;文件导入转 lark-drive,认证/授权转 lark-shared。.

Pinvou/pinvou-agent · 166 tokens