AGENT SKILL

docarray

Use DocArray for multimodal Pydantic-style documents, typed DocList and DocVec batches, serialization and local storage, FastAPI payloads, and vector retrieval indexes.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
docarray
license
Apache 2.0
metadata
{ "disco-role": "operating" }
description
Use DocArray for multimodal Pydantic-style documents, typed DocList and DocVec batches, serialization and local storage, FastAPI payloads, and vector retrieval indexes.
disable-model-invocation
true

02 资源文件 4

03 SKILL.md 正文

DocArray

Use this repo skill when a task names DocArray or asks for a Python data model for multimodal records, typed document batches, document serialization, local document storage, FastAPI document payloads, or vector-index retrieval.

Verified baseline

The verified default is a CPU workflow using DocArray base dependencies plus the proto, pandas, and web extras. Install the public package for normal use:

pip install -U docarray
# add only the surfaces you need:
pip install -U "docarray[proto,pandas,web]"
python -c "import docarray; print(docarray.__version__)"

For the current verified DocVec path, prefer numpy<2 and run the bundled smoke helpers before trusting a new environment. Optional tensor frameworks, media loaders, cloud stores, and external vector databases are not implied by the base install.

Route by task

  • Model a record or multimodal schema: read document-modeling for BaseDoc, predefined modality docs, dynamic schemas, DocList, DocVec, nested fields, and typed tensor shapes.
  • Serialize, store, stream, or serve documents: read serialization-storage for JSON, protobuf, bytes, base64, binary files, CSV/DataFrame exchange, file://, S3 boundaries, and DocArrayResponse.
  • Index, search, filter, or persist vectors: read vector-indexing for InMemoryExactNNIndex, query builders, subindexes, persistence, and optional backend selection.

Tasks that span multiple surfaces should start here, choose the schema in document-modeling, then hand the resulting typed documents to serialization-storage or vector-indexing.

Fast decisions

Need First choice Watch for
One validated data point BaseDoc subclass Required fields, nested docs, and per-document tensor shapes.
Mutable/reorderable/streaming collection DocList[MyDoc] Typed lists are homogeneous; bare DocList may be heterogeneous.
Contiguous ML batch DocVec[MyDoc] Homogeneous fields; optional doc/tensor columns must be all present or all missing.
Human-readable transport JSON Rich tensor/list unions may need explicit schema handling.
Compact trusted transport protobuf or protobuf-array Install docarray[proto]; document unions are not protobuf-safe.
Local retrieval prototype InMemoryExactNNIndex[MyDoc] Dimensioned vector field and backend-specific filter/query behavior.
Production/vector service Optional backend Install and verify the chosen client, service, credentials, schema, and metric separately.

Common failure boundaries

  • Missing google.protobuf, pandas, fastapi, or backend client: install the matching narrow extra; do not install full by default.
  • DocVec fails with a NumPy device error: check NumPy compatibility and try numpy<2 for the current verified CPU path.
  • File-store push raises FileNotFoundError: create the explicit parent namespace directory first.
  • CSV cannot rebuild tensor fields: use JSON, protobuf-array, binary, or DataFrame instead; CSV is safest for scalar rows.
  • In-memory query builder does not support text search, and equal-score result ties can expose a comparison edge case; see the vector troubleshooting reference.

Read references/troubleshooting.md for cross-cutting recovery guidance and references/repo-provenance.md before deciding whether this skill matches a changed checkout.

Routing metadata path convention

In references/repo-routing-metadata.json, every useful_entry_points value is a path relative to this DocArray skill root (the directory containing this SKILL.md), not a repository- or bundle-prefixed path.

Safe bundled checks

These helpers are self-contained and do not require the original repository checkout:

python sub-skills/document-modeling/scripts/schema_smoke.py --help
python sub-skills/serialization-storage/scripts/roundtrip_smoke.py --help
python sub-skills/vector-indexing/scripts/inmemory_index_smoke.py --help
python scripts/check_env.py --help

Run the relevant helper after installing the package and selected extras. Helpers use tiny in-memory or temporary-file fixtures; they do not start databases, use credentials, or download models.

Public capability boundaries

DocArray also exposes Torch, TensorFlow, JAX, image/audio/video/mesh loaders, S3, HNSWLib, Qdrant, Weaviate, Elasticsearch, Redis, Milvus, MongoDB Atlas, Epsilla, and Jina/FastAPI integrations. Those are routed by the sub-skills but remain optional until their exact dependency variant, service/network/credential plan, and native smoke have been verified.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351