tokenization skills

10 tagged tokenization, measured the same way as everything else here.

Browse within: BPE 7Unigram 7

Orchestra-Research/AI-Research-SKILLs

Skill Claude CodeCodex

Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance…

12k +30 2mo ago A 75 tokens original MIT

liortesta/ClawdAgent

Skill Claude CodeCodex

Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance…

11 5d ago A 75 tokens copy · 100% Apache-2.0

canton-enterprise

03

tenzro/tenzro-network

Skill Claude CodeCodex

Canton Network enterprise skill — submit DAML commands, query contracts, manage parties, transfer Canton Coin (CIP-56), create tokenized assets, execute atomic DvP settlements, upload DAR packages.

3 9d ago A 47 tokens original Apache-2.0

ihatesea69/HieuNghi-AI-Skills

Skill Claude CodeCodex

Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance…

3 6mo ago A 75 tokens copy · 100% MIT

bitbadges-builder

05

BitBadges/bitbadges-builder-mcp

Skill Claude CodeCodex

Create BitBadges collections (tokens, NFTs, stablecoins) using MCP tools. Use when user wants to create collections, smart tokens, NFTs, or needs to generate BitBadges transaction JSON.

0 4mo ago A 45 tokens archived