AGENT SKILL

ac-text-encoding

Handling text as bytes without corrupting it: what UTF-8 actually encodes and its malformed forms, normalization forms and why two identical-looking strings differ, grapheme clusters versus code points versus bytes, encoding boundaries at input and output, mojibake and how it happens, and the written text-policy record. It explains how text is represented and where it breaks; it never recommends a library and never guesses an encoding. Use when asking "UTF-8", "encoding error", "mojibake", "why do these strings not match", "normalization form", "NFC versus NFD", "grapheme cluster", "code point", "invalid byte sequence", "text corruption", "why does my hash of the same text differ". Not for locale formatting and language presentation (ac-i18n-and-localisation), not for collation and search (ac-i18n-and-localisation), not for parsing structured data formats (ac-api-design).

License 许可
MIT
最近更新
2026-11-15
Stars
★ 81
Forks
17

01 前言 Front Matter

name
ac-text-encoding
status
seedling
license
MIT
metadata
{ "domain": "software-engineering" }
description
Handling text as bytes without corrupting it: what UTF-8 actually encodes and its malformed forms, normalization forms and why two identical-looking strings differ, grapheme clusters versus code points versus bytes, encoding boundaries at input and output, mojibake and how it happens, and the written text-policy record. It explains how text is represented and where it breaks; it never recommends a library and never guesses an encoding. Use when asking "UTF-8", "encoding error", "mojibake", "why do these strings not match", "normalization form", "NFC versus NFD", "grapheme cluster", "code point", "invalid byte sequence", "text corruption", "why does my hash of the same text differ". Not for locale formatting and language presentation (ac-i18n-and-localisation), not for collation and search (ac-i18n-and-localisation), not for parsing structured data formats (ac-api-design).

02 资源文件 10

03 SKILL.md 正文

Text encoding

Overview

Text in software is bytes with an interpretation, and most text bugs are an interpretation mismatch: bytes written as one encoding and read as another, two strings that render identically but differ underneath because they are built from different character sequences, or a truncation that splits a multi-byte character in half and produces corruption the system then stores. UTF-8 is the default answer, but using it correctly requires knowing what it does not promise — that two visually equal strings are byte-equal.

This skill covers the representation layer beneath localisation: the encoding itself and its malformed forms, the normalisation that makes comparisons honest, the three units that get confused (byte, code point, grapheme cluster), the boundaries where conversion must happen, and the mojibake signatures that identify which mismatch occurred. Presenting text in a language's conventions belongs to the localisation skill; this one is about keeping the bytes intact.

Primary sources: https://www.unicode.org/versions/latest/ and https://www.rfc-editor.org/rfc/rfc3629.

When to use

  • Storing, transmitting, or comparing text and getting mismatches or corruption.
  • Building strings from parts, truncating, or hashing text.
  • Diagnosing mojibake and identifying which encoding confusion produced it.
  • Setting a system-wide text policy.

When NOT to use

  • For locale formats, translations, plural rules (use ac-i18n-and-localisation).
  • For collation and search across scripts (use ac-i18n-and-localisation).
  • For text-heavy corpus or search systems (use ac-retrieval-and-vector-systems).

Subskill Index

Leaf Domain segment Scope (one line)
subskills/utf8-and-malformed-input.md software-engineering/utf8-and-malformed-input The encoding, its byte forms, overlong forms, and rejecting bad input.
subskills/normalization-forms.md software-engineering/normalization-forms NFC and NFD, composed versus decomposed, and choosing one.
subskills/units-of-text.md software-engineering/units-of-text Bytes, code points, and grapheme clusters, and which operation needs which.
subskills/encoding-boundaries.md software-engineering/encoding-boundaries Where conversion happens and the rule for each boundary.
subskills/mojibake-diagnosis.md software-engineering/mojibake-diagnosis Reading the garbage to identify the mismatch that made it.
subskills/text-in-storage.md software-engineering/text-in-storage Column encodings, collation interactions, and lengths that lie.
subskills/text-policy-record.md software-engineering/text-policy-record The written record: encoding, normalization, and boundary rules.

Status and honest limits

Encoding behaviour is correct only where every layer agrees: the source of the bytes, the transport, the store, and the reading code. Historical data predating a policy may never reveal which encoding it was written in, and the best available answer for it is a documented assumption, not a certainty. Nothing recommends a library, and the Unicode Standard itself changes between versions — new characters are assigned, which changes what any fixed version's tables know.

04 同仓库更多 Skills

jiayaoqijia
jiayaoqijia/cryptoskill

troves

Deposit and withdraw from Troves DeFi yield vault strategies on Starknet. View available strategies, check positions, and manage vault deposits. Use this skill when the user mentions Troves, vault, yield vault, DeFi vault, strategy, yield farming, vault deposit, vault withdraw, earning yield through vaults, passive yield strategies, or wants to deposit into or withdraw from a yield strategy — even if they don't say "Troves" explicitly.

★ 81
jiayaoqijia
jiayaoqijia/cryptoskill

trade

Swap tokens on Starknet via Fibrous DEX aggregator. Supports per-provider selection (avnu, ekubo, or auto to race all). Use this skill when the user wants to swap, exchange, trade, convert, buy, or sell one token for another on Starknet. Also use when the user asks about token prices, exchange rates, DEX routing, best swap route, or wants to check how much they would receive for a given amount — even if they don't explicitly say "swap".

★ 81
jiayaoqijia
jiayaoqijia/cryptoskill

staking

Stake, unstake, claim, and compound rewards for STRK, WBTC, tBTC, SolvBTC, and LBTC on Starknet. View validators, pools, and staking positions. Use this skill when the user mentions staking, delegating, validators, staking pools, rewards, compounding, unstaking, earning yield, passive income on crypto, or wants to earn on their STRK or BTC tokens — even if they don't say "stake" explicitly.

★ 81
jiayaoqijia
jiayaoqijia/cryptoskill

send

Send STRK, ETH, or any ERC-20 token to a Starknet address. Supports simulation (dry-run) before execution. Use this skill when the user wants to send, transfer, pay, tip, or move tokens to another wallet address on Starknet. Also trigger when the user mentions sending funds to someone, making a payment, transferring assets, or moving crypto to a specific address — even if they just say "pay 10 STRK to 0x...".

★ 81
jiayaoqijia
jiayaoqijia/cryptoskill

portfolio

View a comprehensive DeFi portfolio dashboard and optimize portfolio allocation via automated rebalancing. Use this skill when the user wants an overview, summary, dashboard, total value, net worth, portfolio rebalancing, target allocation, or wants to see all their DeFi positions across Starknet at once. Also trigger when the user asks "what do I have", "show me everything", "how much am I worth", "my positions", "my investments", "rebalance my portfolio", "I want 50% ETH", "optimize my holdings", or any request for a holistic view of their Starknet holdings — even if they don't explicitly say "portfolio".

★ 81
jiayaoqijia
jiayaoqijia/cryptoskill

multi-swap

Execute multiple token swaps in a single Starknet transaction via Fibrous (default). Supports per-provider selection (avnu, ekubo, or auto to race all). Supports 2-3 swap pairs bundled into one multicall. Use this skill when the user wants to swap multiple token pairs at once, do batch swaps, execute several trades simultaneously, perform parallel swaps, or do bulk trading in one transaction. Also trigger when the user mentions "two swaps", "three swaps", "swap X and Y at the same time", or wants to convert multiple tokens in a single call.

★ 81