AGENT SKILL

rank-bm25

Guides tokenized lexical document retrieval with the rank_bm25 Python package, including BM25Okapi, BM25L, BM25Plus, full-corpus scoring, top-n retrieval, batch scoring, and input troubleshooting.

License 许可
Apache-2.0
最近更新
2026-09-03
Stars
★ 351
Forks
27

01 前言 Front Matter

name
rank-bm25
license
Apache 2.0
metadata
{ "disco-role": "operating" }
description
Guides tokenized lexical document retrieval with the rank_bm25 Python package, including BM25Okapi, BM25L, BM25Plus, full-corpus scoring, top-n retrieval, batch scoring, and input troubleshooting.
disable-model-invocation
true

02 资源文件 6

03 SKILL.md 正文

rank_bm25

Use this skill when a task asks for the rank_bm25 package, Okapi BM25, BM25L, BM25+, sparse lexical ranking, or a small in-memory search index over pre-tokenized documents. It teaches the package API; it is not a production search-service design and it does not cover the commented-out BM25Adpt/BM25T classes.

Route the task

  • Install or import: install the public rank_bm25 distribution, ensure NumPy is available, and run the import check below.
  • Build and rank an index: read workflows.md for a complete tokenization, indexing, scoring, and top-n recipe.
  • Choose an algorithm or use advanced methods: read api-reference.md for verified signatures, defaults, return types, parameter meaning, and batch-score semantics.
  • Diagnose a failed ranking workflow: read troubleshooting.md, then use the bundled smoke_rank_bm25.py helper for a deterministic tiny-fixture check.
  • Check whether this knowledge matches a checkout: read repo-provenance.md before deciding whether the skill needs refreshing.

Minimal setup and import check

python -m pip install rank_bm25
python -c "from rank_bm25 import BM25Okapi, BM25L, BM25Plus; print('rank_bm25 ready')"

The package has no console CLI and its runtime dependency is NumPy. CPU execution is the supported baseline for the covered in-memory workflows.

Core operating contract

  1. Represent every document as a sequence of string tokens, for example document.split(); pass a list of those token sequences as corpus.
  2. Apply the same lowercasing, normalization, stopword handling, and tokenization policy to documents and queries yourself. The package does no text preprocessing.
  3. Choose BM25Okapi, BM25L, or BM25Plus, construct it once for a corpus, and reuse the index for compatible tokenized queries.
  4. Use get_scores(query) for one score per indexed document, get_top_n(query, documents, n=5) when you need the original documents, or get_batch_scores(query, doc_ids) for a selected index subset.
  5. Keep the original documents list in exactly the same order as the corpus; get_top_n checks the lengths and maps score indices back into that list.
  6. Validate one known query and expected top document before scaling the corpus. Run python scripts/smoke_rank_bm25.py from this skill directory or any current working directory after installing the package.

The package is intentionally lightweight and in-memory. For full API details, recipes, and predictable failure recovery, follow the linked references rather than reopening the source repository.

04 同仓库更多 Skills

VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-triton-opt

Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-solve

END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-skill

PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-pattern-fusion

Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-orchestrate

Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.

★ 351
VectorSpaceLab
VectorSpaceLab/AREX-Skill

passnet-feedback

Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).

★ 351