rank-bm25
Guides tokenized lexical document retrieval with the rank_bm25 Python package, including BM25Okapi, BM25L, BM25Plus, full-corpus scoring, top-n retrieval, batch scoring, and input troubleshooting.
- License 许可
- Apache-2.0
- 最近更新
- 2026-09-03
- Stars
- ★ 351
- Forks
- 27
01 前言 Front Matter
- name
- rank-bm25
- license
- Apache 2.0
- metadata
- { "disco-role": "operating" }
- description
- Guides tokenized lexical document retrieval with the rank_bm25 Python package, including BM25Okapi, BM25L, BM25Plus, full-corpus scoring, top-n retrieval, batch scoring, and input troubleshooting.
- disable-model-invocation
- true
02 资源文件 6
- 参考文档references/api-reference.md4.6 KB
- 参考文档references/repo-provenance.md1.7 KB
- 参考文档references/repo-routing-metadata.json333 B
- 参考文档references/troubleshooting.md5.3 KB
- 参考文档references/workflows.md3.2 KB
- 脚本scripts/smoke_rank_bm25.py2.4 KB
03 SKILL.md 正文
rank_bm25
Use this skill when a task asks for the rank_bm25 package, Okapi BM25,
BM25L, BM25+, sparse lexical ranking, or a small in-memory search index over
pre-tokenized documents. It teaches the package API; it is not a production
search-service design and it does not cover the commented-out BM25Adpt/BM25T
classes.
Route the task
- Install or import: install the public
rank_bm25distribution, ensure NumPy is available, and run the import check below. - Build and rank an index: read workflows.md for a complete tokenization, indexing, scoring, and top-n recipe.
- Choose an algorithm or use advanced methods: read api-reference.md for verified signatures, defaults, return types, parameter meaning, and batch-score semantics.
- Diagnose a failed ranking workflow: read troubleshooting.md, then use the bundled smoke_rank_bm25.py helper for a deterministic tiny-fixture check.
- Check whether this knowledge matches a checkout: read repo-provenance.md before deciding whether the skill needs refreshing.
Minimal setup and import check
python -m pip install rank_bm25
python -c "from rank_bm25 import BM25Okapi, BM25L, BM25Plus; print('rank_bm25 ready')"
The package has no console CLI and its runtime dependency is NumPy. CPU execution is the supported baseline for the covered in-memory workflows.
Core operating contract
- Represent every document as a sequence of string tokens, for example
document.split(); pass a list of those token sequences ascorpus. - Apply the same lowercasing, normalization, stopword handling, and tokenization policy to documents and queries yourself. The package does no text preprocessing.
- Choose
BM25Okapi,BM25L, orBM25Plus, construct it once for a corpus, and reuse the index for compatible tokenized queries. - Use
get_scores(query)for one score per indexed document,get_top_n(query, documents, n=5)when you need the original documents, orget_batch_scores(query, doc_ids)for a selected index subset. - Keep the original
documentslist in exactly the same order as the corpus;get_top_nchecks the lengths and maps score indices back into that list. - Validate one known query and expected top document before scaling the corpus.
Run
python scripts/smoke_rank_bm25.pyfrom this skill directory or any current working directory after installing the package.
The package is intentionally lightweight and in-memory. For full API details, recipes, and predictable failure recovery, follow the linked references rather than reopening the source repository.
04 同仓库更多 Skills
passnet-triton-opt
Make a single PassNet Triton kernel fast AND numerically faithful: performance model (when a replacement can win at all), block/grid/warp tuning, autotune policy, launch overhead, and per-op numeric recipes to pass the dtype baseline tolerances. Use when a pass matches and is correct but speedup below expected, or when correctness fails by small numeric margins.
passnet-solve
END-TO-END playbook for solving one PassNet sample: analyze the computation graphs, decide the optimization strategy (what to fuse, what to replace, what to leave alone), drive the iteration loop, and maximize the sample score. This is the ENTRY skill — invoke it first for any PassNet optimization task; it tells you when to use passnet-pattern-fusion, passnet-triton-opt and passnet-feedback.
passnet-skill
PassNet GPU kernel optimization via compiler passes. Design and implement Triton-based optimization passes, create pass files under ./pass_dir/, self-evaluate with pass_evaluator, and iterate to maximize GPU speedup.
passnet-pattern-fusion
Author PassNet pass files: write patterns that actually MATCH the FX graph, pick fusion regions (which ops to absorb into one Triton kernel), and structure multi-pass submissions with the mandatory shared-dispatch architecture. Use when creating or fixing pattern()/replacement_args()/replacement_func() files, when a pass "failed to match", or when deciding how to fuse multiple kernels/ops.
passnet-orchestrate
Round/batch planning and decision support for PassNet graph optimization. Use for multi-sample triage, worker allocation, eval-budget policy, and post-eval keep/revert decisions. For solving one concrete sample end to end, use passnet-solve as the entry skill; consult this skill only when you need broader planning or a specific decision gate.
passnet-feedback
Fast, deterministic feedback for PassNet work: pre-flight pattern/match verification WITHOUT burning a GPU evaluation, per-node bottleneck analysis of a sample's graphs, and eval-log parsing into per-variant status + estimated score + failure classification. Use BEFORE every GPU evaluation (check_pattern), at the START of a sample (analyze_graph), and AFTER every evaluation (parse_eval_log).