AGENT SKILL

special-tokens

Explains tokenizer control sequences such as <|endoftext|> and <|im_start|>. Use when the user asks about tokenizer internals.

License 许可
Apache-2.0
最近更新
2026-08-17
Stars
★ 3
Forks
2

01 前言 Front Matter

name
special-tokens
description
Explains tokenizer control sequences such as <|endoftext|> and <|im_start|>. Use when the user asks about tokenizer internals.

03 SKILL.md 正文

Body

The model stops at <|endoftext|>.

04 同仓库更多 Skills