[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6BZ2N9X2VT23V2P7V6KJ14V":3,"skill-same:01M6BZ2N9X2VT23V2P7V6KJ14V":74},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":39,"linkOnly":40,"markdownContent":41,"markdownBody":42,"frontMatter":43,"refName":44,"commitSha":45,"skillDirectory":46,"forksCount":47,"repositoryHtmlUrl":48,"repositoryHomepageUrl":49,"ownerHtmlUrl":50,"ownerBlogUrl":51,"ownerDescription":52,"updatedAt":53,"install":54,"resources":67},"01M6BZ2N9X2VT23V2P7V6KJ14V","tree-sitter-language-pack-chunking-for-llms",null,"Use when the user wants to split source code into chunks for an LLM context window without breaking syntax mid-construct. Covers `ts-pack process --chunk-size`, why syntax-aware splits beat fixed-byte splits, picking a size, and the chunk JSON shape.","xberg-io\u002Ftree-sitter-language-pack","xberg-io","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F241328462?v=4","01M6BYYMACFMK5WNT338Y71K58","Xberg.io","01M6BYYMEGM63ZCEYZJRTVS9JT",485,"MIT","https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack\u002Fblob\u002Fmain\u002Fplugin\u002F.pi\u002Fskills\u002Ftree-sitter-language-pack-chunking-for-llms\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"tree sitter language pack","pack","language","for","tree","chunking","tree sitter","language pack","llms","tree sitter language pack chunking for llms","code","chunk","chunks","window","when","why","syntax","source","source code","context","2026-11-10 10:12:30",false,"---\ndescription: Use when the user wants to split source code into chunks for an LLM context window without breaking syntax mid-construct. Covers `ts-pack process --chunk-size`, why syntax-aware splits beat fixed-byte splits, picking a size, and the chunk JSON shape.\nname: tree-sitter-language-pack-chunking-for-llms\n---\n\n\u003C!--\nAI-RULEZ :: GENERATED FILE — DO NOT EDIT\nContent-Hash: blake3:f73428c23ccd047c1e8d74065c6be1dcc21b97f1f3e235ebc2a6779317208188\nSource-Hash: blake3:b8687cf31ac8c66a5ecc15c4f11520ff201d8db14467b7528d5120124e9753a4\nSchema-Version: v1\n-->\n\n# Syntax-aware chunking for LLMs\n\nSplitting code on a fixed byte or line count cuts functions in half and\nstrips context. `ts-pack process \u003Cfile> --chunk-size \u003Cbytes>` splits on\nsyntactic boundaries (whole functions, classes, blocks) so each chunk is a\ncoherent unit, and emits them in the JSON `chunks` array.\n\n## Quick recipe\n\n```bash\n# ~2 KB chunks aligned to syntax boundaries\nts-pack process src\u002Fapp.ts --chunk-size 2000\n```\n\n`--chunk-size` is a maximum size in bytes. The splitter packs whole\nsyntactic units up to that bound; an oversized single construct becomes its\nown chunk rather than being cut. Chunks are added to the normal `process`\nJSON output under `chunks`.\n\n## Picking a size\n\n- Match the downstream model's token budget. A rough rule: bytes ÷ 4 ≈\n  tokens for code, so `--chunk-size 4000` is on the order of ~1k tokens.\n- Larger chunks preserve more local context but fit fewer per request.\n- Leave headroom for the prompt, the surrounding messages, and the\n  response — do not size chunks to the full context window.\n\n## Combining with extraction\n\nChunking composes with the other `process` features, so you can attach\nstructure metadata to each request:\n\n```bash\nts-pack process src\u002Fservice.py --structure --chunk-size 3000 \\\n  | jq '{chunks: (.chunks | length), functions: (.structure | length)}'\n```\n\n## Chunk output\n\n`chunks` is a list of code-chunk objects in the `process` JSON. Each chunk\ncarries its source text plus span information (line\u002Fbyte offsets), so you\ncan cite or re-locate a chunk back in the original file. Iterate the array\nto feed an LLM one coherent unit at a time:\n\n```bash\nts-pack process big_module.py --chunk-size 2500 \\\n  | jq -c '.chunks[]'\n```\n\n## SDK equivalent\n\nThe SDK exposes chunking through the process config: set the\n`chunk_max_size` field (in bytes) on `ProcessConfig` — the same value the\nCLI's `--chunk-size` flag sets. `ProcessConfig` is a frozen dataclass, so\npass it to the constructor:\n\n```python\nfrom tree_sitter_language_pack import process, ProcessConfig\n\nconfig = ProcessConfig(\"python\", chunk_max_size=2500)\nresult = process(source_code, config)\nfor chunk in result.chunks:          # ProcessResult is an object, not a dict\n    send_to_llm(chunk.content)\n```\n\n## When not to chunk\n\nFor a single small file that already fits the context window, skip\nchunking and pass the file whole. Reach for chunking when a file is large,\nwhen you are batching many files into a RAG index, or when you need stable,\nsyntactically coherent units to cite.\n","\u003C!--\nAI-RULEZ :: GENERATED FILE — DO NOT EDIT\nContent-Hash: blake3:f73428c23ccd047c1e8d74065c6be1dcc21b97f1f3e235ebc2a6779317208188\nSource-Hash: blake3:b8687cf31ac8c66a5ecc15c4f11520ff201d8db14467b7528d5120124e9753a4\nSchema-Version: v1\n-->\n\n# Syntax-aware chunking for LLMs\n\nSplitting code on a fixed byte or line count cuts functions in half and\nstrips context. `ts-pack process \u003Cfile> --chunk-size \u003Cbytes>` splits on\nsyntactic boundaries (whole functions, classes, blocks) so each chunk is a\ncoherent unit, and emits them in the JSON `chunks` array.\n\n## Quick recipe\n\n```bash\n# ~2 KB chunks aligned to syntax boundaries\nts-pack process src\u002Fapp.ts --chunk-size 2000\n```\n\n`--chunk-size` is a maximum size in bytes. The splitter packs whole\nsyntactic units up to that bound; an oversized single construct becomes its\nown chunk rather than being cut. Chunks are added to the normal `process`\nJSON output under `chunks`.\n\n## Picking a size\n\n- Match the downstream model's token budget. A rough rule: bytes ÷ 4 ≈\n  tokens for code, so `--chunk-size 4000` is on the order of ~1k tokens.\n- Larger chunks preserve more local context but fit fewer per request.\n- Leave headroom for the prompt, the surrounding messages, and the\n  response — do not size chunks to the full context window.\n\n## Combining with extraction\n\nChunking composes with the other `process` features, so you can attach\nstructure metadata to each request:\n\n```bash\nts-pack process src\u002Fservice.py --structure --chunk-size 3000 \\\n  | jq '{chunks: (.chunks | length), functions: (.structure | length)}'\n```\n\n## Chunk output\n\n`chunks` is a list of code-chunk objects in the `process` JSON. Each chunk\ncarries its source text plus span information (line\u002Fbyte offsets), so you\ncan cite or re-locate a chunk back in the original file. Iterate the array\nto feed an LLM one coherent unit at a time:\n\n```bash\nts-pack process big_module.py --chunk-size 2500 \\\n  | jq -c '.chunks[]'\n```\n\n## SDK equivalent\n\nThe SDK exposes chunking through the process config: set the\n`chunk_max_size` field (in bytes) on `ProcessConfig` — the same value the\nCLI's `--chunk-size` flag sets. `ProcessConfig` is a frozen dataclass, so\npass it to the constructor:\n\n```python\nfrom tree_sitter_language_pack import process, ProcessConfig\n\nconfig = ProcessConfig(\"python\", chunk_max_size=2500)\nresult = process(source_code, config)\nfor chunk in result.chunks:          # ProcessResult is an object, not a dict\n    send_to_llm(chunk.content)\n```\n\n## When not to chunk\n\nFor a single small file that already fits the context window, skip\nchunking and pass the file whole. Reach for chunking when a file is large,\nwhen you are batching many files into a RAG index, or when you need stable,\nsyntactically coherent units to cite.\n",{"name":5,"description":7},"main","6403d3051157d3808dcdca49aeb0c211742d1d61","plugin\u002F.pi\u002Fskills\u002Ftree-sitter-language-pack-chunking-for-llms",71,"https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack","https:\u002F\u002Fdocs.tree-sitter-language-pack.xberg.io","https:\u002F\u002Fgithub.com\u002Fxberg-io","https:\u002F\u002Fxberg.io","Open-source content intelligence with a Rust core: documents, web, LLMs and code, with native bindings for every major language.","2026-11-07 18:08:38",{"source":8,"skillName":5,"commands":55,"prompt":65,"zipAvailable":66},[56,59,62],{"tool":57,"command":58},"npx","npx skills add xberg-io\u002Ftree-sitter-language-pack --skill tree-sitter-language-pack-chunking-for-llms",{"tool":60,"command":61},"bunx","bunx skills add xberg-io\u002Ftree-sitter-language-pack --skill tree-sitter-language-pack-chunking-for-llms",{"tool":63,"command":64},"pnpm","pnpm dlx skills add xberg-io\u002Ftree-sitter-language-pack --skill tree-sitter-language-pack-chunking-for-llms","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack\u002Fblob\u002Fmain\u002Fplugin\u002F.pi\u002Fskills\u002Ftree-sitter-language-pack-chunking-for-llms\u002FSKILL.md",true,[68],{"path":69,"resourceType":70,"resourceTypeDesc":71,"sizeBytes":72,"sourceHtmlUrl":73},"index.md","REFERENCE","参考文档",566,"https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack\u002Fblob\u002Fmain\u002Fplugin\u002F.pi\u002Fskills\u002Ftree-sitter-language-pack-chunking-for-llms\u002Findex.md",[75,97,121,136,154,169],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":86,"publishedAt":39},"01M6BZ48YBWAZH9X1XZ6QVA5TD","tree-sitter-language-pack","Parse and extract code intelligence from 371 programming languages using tree-sitter grammars. Use when writing code that parses source, extracts structure\u002Fimports\u002Fexports\u002Fsymbols\u002Fdocstrings\u002Fcomments, detects a language, runs syntax diagnostics, or produces syntax-aware chunks for LLMs — in Rust, Python, Node.js\u002FTypeScript, or the ts-pack CLI. Covers installation, the CLI surface, the SDK surface, and parser-cache management.","https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack\u002Fblob\u002Fmain\u002Fskills\u002Ftree-sitter-language-pack\u002FSKILL.md",[81],{"categoryNo":82,"categoryName":83,"taxonomyType":84,"taxonomyTypeDesc":85},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[19,26,25,23,21,20,87,88,29,27,89,90,91,92,93,94,31,95,96,33],"rust","writing","cli","node","parser","typescript","io","structure","imports","parse",{"skillNo":98,"name":99,"nameZh":6,"description":100,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":101,"categories":102,"keywords":111,"publishedAt":39},"01M6BZ461Q0A0TZJG56G6J5QF0","tree-sitter-language-pack-using-the-mcp-server","Use when parsing source, extracting code structure, or detecting a language through the tree-sitter-language-pack MCP server's tools, rather than shelling out to the ts-pack CLI. Covers the tool surface, the auto-installing launcher, and when MCP beats the CLI or SDK.","https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack\u002Fblob\u002Fmain\u002Fskills\u002Ftree-sitter-language-pack-using-the-mcp-server\u002FSKILL.md",[103,108],{"categoryNo":104,"categoryName":105,"taxonomyType":106,"taxonomyTypeDesc":107},"01JBSEEDCAT000000000000010","后端与接口","SCENARIO","使用场景",{"categoryNo":109,"categoryName":110,"taxonomyType":106,"taxonomyTypeDesc":107},"01JBSEEDCAT000000000000023","Skill与智能体开发",[112,113,19,114,115,26,25,23,21,20,29,89,116,117,118,119,94,36,120,33],"tree sitter language pack using the mcp server","MCP","mcp server","server","detecting","extracting","launcher","auto","Tools",{"skillNo":122,"name":123,"nameZh":6,"description":124,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":125,"categories":126,"keywords":127,"publishedAt":39},"01M6BZ437CE3DZQ22GYHMNMMVK","tree-sitter-language-pack-parsing-source","Use when the user wants a tree-sitter syntax tree for a source file — an s-expression dump or JSON tree. Covers `ts-pack parse`, language auto-detection vs `--language`, stdin input, and reading `has_errors`.","https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack\u002Fblob\u002Fmain\u002Fskills\u002Ftree-sitter-language-pack-parsing-source\u002FSKILL.md",[],[128,129,19,26,25,23,36,21,20,130,131,35,22,132,133,119,134,135,96,33],"parsing","tree sitter language pack parsing source","file","json","errors","expression","use","reading",{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":140,"categories":141,"keywords":142,"publishedAt":39},"01M6BZ40QMB2C6QF2N09MM15GF","tree-sitter-language-pack-managing-parsers","Use when the user needs to manage the tree-sitter parser cache — prefetch parsers for offline or CI runs, list what is downloaded, inspect a language, find the cache directory, or clean it. Covers `ts-pack download`, `list`, `info`, `cache-dir`, `clean`, and `init`.","https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack\u002Fblob\u002Fmain\u002Fskills\u002Ftree-sitter-language-pack-managing-parsers\u002FSKILL.md",[],[143,144,19,26,25,23,21,20,145,146,147,148,149,22,150,151,152,134,153,33],"parsers","tree sitter language pack managing parsers","ci","init","prefetch","cache","find","inspect","manage","clean","info",{"skillNo":155,"name":156,"nameZh":6,"description":157,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":158,"categories":159,"keywords":161,"publishedAt":39},"01M6BZ3Y1E46H4CPXVJD6SB03D","tree-sitter-language-pack-extracting-code-structure","Use when the user wants structured code metadata from a source file — functions, classes, imports, exports, symbols, docstrings, comments, or syntax diagnostics. Covers `ts-pack process` feature flags, the JSON result shape, and the default feature set.","https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack\u002Fblob\u002Fmain\u002Fskills\u002Ftree-sitter-language-pack-extracting-code-structure\u002FSKILL.md",[160],{"categoryNo":82,"categoryName":83,"taxonomyType":84,"taxonomyTypeDesc":85},[162,19,29,26,25,117,94,23,21,20,163,164,165,35,166,167,36,95,168,33],"tree sitter language pack extracting code structure","extracting code structure","code structure","SET","default","flags","result",{"skillNo":170,"name":171,"nameZh":6,"description":172,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":173,"categories":174,"keywords":175,"publishedAt":39},"01M6BZ3VAJEY26GESPCWQ2JSXA","tree-sitter-language-pack-detecting-languages","Use when the user wants to know which programming language a file or snippet is. Covers implicit detection in `ts-pack parse`\u002F`process`, confirming support with `ts-pack list`\u002F`info`, and the SDK detection functions for path, extension, and raw content.","https:\u002F\u002Fgithub.com\u002Fxberg-io\u002Ftree-sitter-language-pack\u002Fblob\u002Fmain\u002Fskills\u002Ftree-sitter-language-pack-detecting-languages\u002FSKILL.md",[],[176,19,26,25,23,116,21,20,177,130,178,179,180,22,134,181,182,153,96,33],"tree sitter language pack detecting languages","programming language","programming","support","which","functions","with"]