partitioning
Partition local files, file-like objects, URLs, text, HTML, PDF/image, Office, tabular, JSON/NDJSON, XML, RST/MD/EPUB, email, and audio inputs into unstructured Element objects.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- partitioning
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Partition local files, file-like objects, URLs, text, HTML, PDF/image, Office, tabular, JSON/NDJSON, XML, RST/MD/EPUB, email, and audio inputs into unstructured Element objects.
- disable-model-invocation
- true
02 资源文件 5
- 参考文档references/api-reference.md9.0 KB
- 参考文档references/troubleshooting.md5.6 KB
- 参考文档references/workflows.md5.8 KB
- 脚本scripts/inspect_tables.py6.2 KB
- 脚本scripts/partition_diagnostics.py6.9 KB
03 SKILL.md 正文
Partitioning
Use this sub-skill when a task asks an agent to turn source documents into unstructured.documents.elements.Element objects or to diagnose why a document type cannot be partitioned. Route serialization/export of elements to elements-and-metadata, post-partition chunking to chunking, embeddings to embeddings, and extraction-quality scoring to evaluation.
Start Here
- Prefer
unstructured.partition.auto.partition()when the corpus has mixed file types or when MIME/extension detection is acceptable. - Prefer document-specific
partition_*APIs when a format has important knobs, such as PDF/image strategy, HTML parser options, email attachment handling, or spreadsheet/table behavior. - Run
python -m unstructured.cli doctororunstructured doctorbefore debugging optional dependency failures; use--for pdf,--for image,--for doc,--for audio, or--file PATHfor targeted checks. - Use local partition APIs for offline processing. Treat API workflows as network/credential-dependent and do not assume they are available in a normal local agent session.
Core Local API
from unstructured.partition.auto import partition
elements = partition(filename="docs/report.pdf", strategy="auto")
partition() accepts exactly one of filename, file, or url. For file-like objects, open in binary mode and pass metadata_filename when extension-based detection or element metadata should reflect a logical name:
with open("docs/report.pdf", "rb") as file:
elements = partition(file=file, metadata_filename="report.pdf", strategy="fast")
Important options:
content_type: overrides or supplements detection when the MIME type is known.encoding: resolves non-UTF text/HTML/JSON decoding issues.headers,ssl_verify,request_timeout: apply only whenurlis used; set a timeout explicitly for unreliable endpoints.languages: preferred language/OCR hint list, e.g.languages=["eng", "spa"]or ISO codes that can map to Tesseract codes.detect_language_per_element: stores language metadata per element instead of once per document.skip_infer_table_types: controls table structure inference by file type; default skips PDF and common image types.starting_page_number: shifts page metadata when partitioning a PDF section.
Format Routes
- Text:
from unstructured.partition.text import partition_text; accepts exactly one offilename,file, ortext, plusparagraph_grouper. - HTML:
from unstructured.partition.html import partition_html; acceptsfilename,file,text, orurl; useful knobs includeskip_headers_and_footers,html_parser_version,image_alt_mode, and URL request options. - PDF:
from unstructured.partition.pdf import partition_pdf; choosestrategy, table/image/form options, page numbering, password, and pdfminer margins. - Image:
from unstructured.partition.image import partition_image; usestrategy="hi_res"or"ocr_only";"fast"is invalid for images. - Office: use
partition_docx,partition_doc,partition_pptx,partition_ppt,partition_xlsx, andpartition_odtfor format-specific controls. - Email: use
partition_emailfor.emlandpartition_msgfor Outlook.msg; inspect attachment options before recursively partitioning attachments. - Tabular: use
partition_csv,partition_tsv, orpartition_xlsx; table elements can containmetadata.text_as_htmlwhen table structure is inferred. - Markup/books: use
partition_md,partition_rst,partition_org,partition_xml, andpartition_epubfor explicit handling. - JSON streams:
partition_jsonexpects an unstructured-style JSON array of element dictionaries;partition_ndjsonhandles newline-delimited element records. - Audio:
partition_audiotranscribes audio to elements and requires the audio extra plusffmpeg.
See references/api-reference.md for a compact API and dependency matrix.
PDF/Image Strategy Rules
Use strategy="fast" for extractable PDF text when layout detection, OCR, image blocks, forms, or table HTML are not needed. Use strategy="hi_res" when table structure, coordinates, images, layout segmentation, or form extraction matters. Use strategy="ocr_only" for scanned PDFs/images when OCR text is enough and layout segmentation is unnecessary. Use strategy="auto" when the agent should let unstructured pick and fallback based on text extractability and installed dependencies.
auto chooses PDF hi_res if table/image extraction is requested, PDF fast when text is extractable, PDF ocr_only when text is not extractable, and image hi_res by default. If unstructured_inference or unstructured_pytesseract is missing, unstructured may warn and fall back between hi_res, ocr_only, and fast; if no viable path exists for a non-extractable PDF, it raises an error.
Prefer languages over deprecated ocr_languages. Prefer hi_res_model_name over deprecated model_name. Prefer skip_infer_table_types over deprecated pdf_infer_table_structure.
Diagnostics Scripts
Bundled helpers live under this sub-skill's scripts/ directory:
python scripts/partition_diagnostics.py --for pdf
python scripts/partition_diagnostics.py --file docs/report.pdf
python scripts/partition_diagnostics.py --partition docs/report.pdf --strategy fast --limit 5
python scripts/inspect_tables.py output/elements.json --out-dir tables --index
partition_diagnostics.py wraps the public doctor/capability APIs and can run a small local partition preview when the environment supports the target format. inspect_tables.py extracts Table element metadata.text_as_html values from exported JSON into standalone HTML files; it is a portable Python adaptation of the repository's user table-inspection shell script.
Workflow Pointers
- For mixed corpora, inventory extensions first, install only required extras, run doctor for risky families, then partition with
partition()and per-format overrides only where needed. - For PDF table extraction, use
strategy="hi_res"and setskip_infer_table_types=[]or callpartition_pdf(..., infer_table_structure=True); inspect table HTML withscripts/inspect_tables.pyafter serializing elements. - For URL partitioning, always set
request_timeout; for HTML pages preferpartition_html(url=...)if the URL is known to return HTML. - For API partitioning, require an endpoint, credentials, and network access; do not embed credentials in scripts or skill content.
Evidence
This sub-skill is grounded in the package metadata, public README guidance, partitioning modules, file type utilities, doctor/CLI diagnostics, partition behavior tests, example document fixtures, and user helper scripts distilled into this bundled guidance.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).