recsys-pipeline-architect
Designs composable recommendation, ranking, and feed pipelines using the six-stage Source→Hydrator→Filter→Scorer→Selector→SideEffect framework
- License 许可
- MIT
- 最近更新
- 2026-10-22
- Stars
- ★ 1
- Forks
- 0
01 前言 Front Matter
- name
- recsys-pipeline-architect
- risk
- safe
- tags
- [ "recommender-system", "ranking", "feed-algorithm", "recsys", "personalization", "for-you-feed", "rag-reranker", "pipeline-architecture" ]
- tools
- [ "claude", "codex", "cursor", "gemini", "opencode", "cline", "continue", "windsurf" ]
- author
- mturac
- source
- community
- license
- MIT
- category
- data-ai
- date_added
- 2026-05-16
- description
- Designs composable recommendation, ranking, and feed pipelines using the six-stage Source→Hydrator→Filter→Scorer→Selector→SideEffect framework
- source_repo
- mturac/recsys-pipeline-architect
- source_type
- community
- license_source
- https://github.com/mturac/recsys-pipeline-architect/blob/main/LICENSE
03 SKILL.md 正文
recsys-pipeline-architect
Overview
A spec-and-scaffold skill for building composable recommendation, ranking, and feed pipelines. It encodes the six-stage Source → Hydrator → Filter → Scorer → Selector → SideEffect framework popularized by xAI's open-sourced For You algorithm (Apache 2.0). This skill is an independent reimplementation of the pattern — no code is copied from the original — licensed MIT. Use it whenever you need "the top K items for a (user, context)": social feeds, content CMSs, RAG rerankers, task prioritizers, notification triage, search reranking, ad ranking.
When to Use This Skill
- Use when the user wants to build any system that picks "the top K items for a user/context"
- Use when the user asks "how should I rank X" or describes a feed/personalization problem
- Use when the user has a scoring function and needs the pipeline plumbing around it
- Use when the user wants to migrate from a single relevance score to multi-action prediction with tunable weights
- Use when the user is wrapping an LLM/ML scorer and needs filters, hydrators, side-effects, and a runnable scaffold in their stack (TypeScript / Go / Python)
How It Works
Step 1: Clarify the use case
Ask the user three questions (only what is missing):
- What are the items being ranked? (posts, products, tasks, alerts, documents...)
- What is the input context? (user ID, search query, current document, time window...)
- What language / runtime? (TypeScript/Node, Go, Python, Rust...)
Step 2: Walk the eight steps of the spec
The full SKILL walks through: clarify use case → identify candidate sources → list required hydrations → list filters → design scorer chain → selector → side effects → generate scaffold. Each step surfaces the architectural trade-offs (multi-action vs single-score, candidate isolation vs joint scoring, online vs offline batch) so the user makes them explicitly rather than defaulting silently.
Step 3: Emit a runnable scaffold
The upstream repository ships three runnable example scaffolds — every one green on its test suite:
- Strapi v5 plugin (TypeScript, Jest, 3/3 pass) — adds
GET /api/feed/for-youwith multi-action scoring and author diversity - Zentra-compatible pipeline (Go with generics, 3/3 pass) — engine.Module-compatible, standalone-usable
- PMAI task prioritizer (Python / FastAPI / pytest, 3/3 pass) —
GET /tasks/next?user_id=42&limit=10
When the user's stack doesn't match, the skill generates from scratch following the interface definitions in references/interfaces.md (TypeScript, Go, Python, Rust).
Examples
Example 1: Strapi content feed
User: "I'm running a Strapi v5 instance with 50k articles. I want a 'for you' feed personalized to each logged-in user based on their reading history."
Skill walks through the 8 steps, generates a Strapi plugin scaffold using the Strapi example as the template.
Example 2: RAG retrieval reranker
User: "My RAG returns top-50 chunks from a vector DB. I want to rerank them with a more expensive scorer and return top-5."
Skill recognizes this as a single-source pipeline with a scorer chain (cheap retrieval + expensive rerank). Generates a Python async pipeline.
Example 3: Notification triage
User: "We send too many notifications. I want a daily digest that picks the top 10 from the last 24h queue."
Skill identifies this as an offline-batch pipeline. Generates a scheduled job scaffold.
Best Practices
- ✅ Surface the multi-action vs single-score trade-off explicitly — don't default silently
- ✅ Order filters by cost (cheap before expensive); universal filters before user-specific
- ✅ Wrap side effects in fire-and-forget patterns (goroutines / promises without await / asyncio tasks) — never block the response
- ✅ Keep scoring deterministic and cacheable; do diversity reranking as a separate stage
- ✅ Attribute the pattern as "popularized by xAI's open-sourced For You algorithm" when generating output
- ❌ Don't invent benchmark or latency numbers — say "depends on workload, run it yourself"
- ❌ Don't name the user's generated artifact "X-like" or use "For You" branding — the pattern is free, the brand is not
- ❌ Don't conflate this with model architecture: this skill is pipeline plumbing around the scorer, not the scorer itself
Limitations
- This skill scaffolds pipeline plumbing; it does not train ML models — the scoring function is the user's responsibility
- It does not operate deployed pipelines (no monitoring, no autoscaling decisions)
- It does not predict pipeline performance (depends on data, hardware, traffic)
- It does not choose infrastructure (vector DB, cache, queue) — those are outside scope
Security & Safety Notes
- The generated scaffolds are framework code, not application logic — no shell commands, no network fetches, no credential handling
- Filters in the generated cookbook include eligibility/paywall/geo-restriction checks; the skill recommends putting these before scoring (so blocked content is never scored)
- Side-effect stages are always async / fire-and-forget; the skill documents this explicitly in the generated README to prevent users from accidentally blocking the response with cache writes or event emissions
Common Pitfalls
-
Problem: Single-score model gets overfit to one metric (clicks) and degrades on others (long sessions, retention) Solution: Skill recommends multi-action prediction with tunable weights — change behavior by changing weights, no retraining
-
Problem: Joint scoring (transformer over the whole batch) is non-deterministic and uncacheable Solution: Skill defaults to candidate isolation via attention masking; recommends joint only when there's a specific reason (e.g., batch-aware diversity)
-
Problem: Side effects (cache writes, impression emits) block the response Solution: Skill generates fire-and-forget patterns and documents the constraint
Upstream
This skill is a thin adapter to the upstream repository. For the full SKILL.md content, 5 reference documents (interfaces in 4 languages, multi-action scoring, candidate isolation, filter cookbook, scorer cookbook), and 3 runnable example scaffolds with passing test suites:
- Repository: https://github.com/mturac/recsys-pipeline-architect
- Release: v0.1.0
- Install via skills.sh:
npx skills add mturac/recsys-pipeline-architect - Pattern source: https://github.com/xai-org/x-algorithm (Apache 2.0; this skill is MIT)
04 同仓库更多 Skills
testing-best-practices
Laravel test design and review. Use when selecting coverage, naming or structuring tests, choosing assertions or test data, isolating dependencies, testing HTTP or security boundaries, improving suite performance, or reviewing test value. Use framework guidance or search-docs for Pest and PHPUnit syntax.
tailwindcss-development
Always invoke when the user's message includes 'tailwind' in any form. Also invoke for: building responsive grid layouts (multi-column card grids, product grids), flex/grid page structures (dashboards with sidebars, fixed topbars, mobile-toggle navs), styling UI components (cards, tables, navbars, pricing sections, forms, inputs, badges), adding dark mode variants, fixing spacing or typography, and Tailwind v3/v4 work. The core use case: writing or fixing Tailwind utility classes in HTML templates (Blade, JSX, Vue). Skip for backend PHP logic, database queries, API routes, JavaScript with no HTML/CSS component, CSS file audits, build tool configuration, and vanilla CSS.
laravel-best-practices
Apply this skill whenever writing, reviewing, or refactoring Laravel PHP code. This includes creating or modifying controllers, models, migrations, form requests, policies, jobs, scheduled commands, service classes, and Eloquent queries. Triggers for N+1 and query performance issues, caching strategies, authorization and security patterns, validation, error handling, queue and job configuration, route definitions, and architectural decisions. Also use for Laravel code reviews and refactoring existing Laravel code to follow best practices. Covers any task involving Laravel backend PHP code patterns.
infer-conventions
Use this skill to analyze how a Laravel application is actually written and record its conventions as shared rules. Trigger when the user wants to detect, infer, document, or standardize project conventions or coding style, set up or grow `.ai/rules`, resolve mixed or conflicting patterns (e.g. "are we using Form Requests or inline validation?"), or onboard agents and teammates to "how we do things here". Covers: a systematic sweep of ~49 Laravel convention dimensions (validation, models, architecture, testing, frontend, database, console), open-ended house-pattern discovery, conflict reporting, and recording rules scoped to the right paths via the Boost `record-rule` MCP tool. Only run this skill when the user explicitly asks for it; never start a sweep as part of another task. Do not use for one-off code review, enforcing formatting a linter already handles, or editing `.ai/rules` files by hand.
inertia-react-development
Develops Inertia.js v2 React client-side applications. Activates when creating React pages, forms, or navigation; using <Link>, <Form>, useForm, or router; working with deferred props, prefetching, or polling; or when user mentions React with Inertia, React pages, React forms, or React navigation.
deploying-to-cloud
Deploys and manages Laravel applications on Laravel Cloud using the `cloud` CLI. Use when the user wants to deploy an app, ship to Cloud, create or manage applications, environments, databases, caches, object storage, queues, domains, instances, background processes, secrets, compute, scheduled tasks, or Laravel Cloud infrastructure; or check billing, usage, or spend. Triggers on deployment, shipping, Cloud management, environment setup, database provisioning, billing or usage queries, the `cloud` CLI, and Laravel Cloud deployment troubleshooting.