oss-project-evaluation
Use when reviewing an OSS project: usage, pros/cons, adopt.
- License 许可
- MIT
- 最近更新
- 2026-10-20
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- oss-project-evaluation
- author
- Hermes Agent
- license
- MIT
- version
- 1.0.0
- metadata
- { "hermes": { "tags": [ "oss", "evaluation", "due-diligence", "github", "adoption", "security-review" ], "category": "software-development", "related_skills": [ "github", "codebase-onboarding" ] } }
- platforms
- [ "linux", "macos", "windows" ]
- description
- Use when reviewing an OSS project: usage, pros/cons, adopt.
02 资源文件 2
- 参考文档references/claim-verification-checklist.md4.5 KB
- 参考文档references/secrets-tool-evaluation.md4.4 KB
03 SKILL.md 正文
OSS project evaluation (用法 / 优劣 / adopt-or-not)
Review a third-party open-source project the way this user consumes it: how do I use it, what is it actually good for, what are its verified risks — not a brochure summary. The deliverable is the verdict, and every strength/weakness claim in it is something you checked yourself.
When to Use
- 用户发来一个 repo 或项目名,问「用法」「优劣」「怎么样」「值不值得用」「要不要用」。
- 用户先让搜索再跟进评估("github 搜索 X" → 追问其中某个项目)。
- 用户带质量要求要替代品("github 搜索 X,安全模型要好")—— 交付的是对比短名单, 不是单项目评估。
- 用户对已评估过的工具继续追问单一维度(跨平台兼容性、agent 交互适应性等)—— 交付 同一短名单上的逐项对比矩阵,不重新做仓库发现。
- A candidate project needs an adoption verdict before the user invests time, credentials, or an agent integration in it.
Workflow
- Identify the repo. If only a name/topic was given, find it first — search breadth
tips (multi-language queries, shortlist enrichment) live in the
githubskill's repo-discovery reference. - Metadata pass, no clone yet: license, stars, archived, last push, language; releases (cadence + which OS assets actually ship); contributors (bus factor); recent commit subjects. Commands: same github reference.
- Shallow-clone into the Hermes scratch dir and read the README IN FULL — READMEs run
past 15k chars; page with read_file offsets. Marketing tables, security claims and
platform promises are the things step 4 will test.
Clone pattern:
GIT_TERMINAL_PROMPT=0 git -c credential.helper='' clone --depth 1 <url> <dir>withworkdirset to the scratch dir (avoids MSYS path translation on Windows). - Claim verification in source — the core of the job. For each headline claim, read
the code that implements it: crypto/KDF and key material, where secrets are stored,
what the binary talks to on the network, permission handling, feature/platform
gating. When claim and code disagree, the code wins — cite
path.rs:line. Checklist:references/claim-verification-checklist.md. - Platform reality check against THIS machine (user is on Windows): does the installer
branch for Windows at all, are permission hardenings
#[cfg(unix)]no-ops, are agent-integration paths hardcoded to unix locations (~/.hermesvs the real%LOCALAPPDATA%\hermes, active profiledefaultnotfree)? Report what actually works here, including the install path you would use on this OS. - Deliver in order: 用法 (concrete commands in first-use order) → 优势 (verified) → 劣势/风险 (verified) → 结论 (worth using for X / not for Y; harder-model alternatives when the domain is secrets/credentials).
- Criterion-driven alternative shortlist (user names a quality bar like "安全模型要好",
usually after an earlier review): group the field by the MECHANISM that decides the
criterion — for secrets, the KEK/trust boundary (OS keystore / passphrase+KDF /
hardware / cloud vault; "self-made key" is its own group) — give one verified evidence
line per candidate, list explicitly what you did NOT verify, and tie each group back to
the weaknesses found in the earlier review. Search hits alone rank low-star newcomers:
add known-name mature projects and say which candidates came from where. Domain depth
for secrets/credential tooling (all three axes):
references/secrets-tool-evaluation.md. - Axis drill-down rounds (跨平台兼容性, agent 交互适应性, …): answer each new axis as a
one-row-per-tool matrix over the SAME shortlist — no re-discovery. Reuse the scratch
reads dir (fetched READMEs, release asset lists, doc pages) so a new round only pays
for its axis-specific fetches; every cell keeps the step-4 evidence standard. Per-axis
procedures:
references/secrets-tool-evaluation.md.
Output contract
- Chinese, plain text — the CLI renders no markdown; no headers/bold/fences, layout via blank lines, dashes and indentation only.
- Evidence per non-obvious claim:
file.rs:line, an exactgh apifield, or command output. "README says X" alone is half a statement — pair it with what the code does. - State openly when marketing is overstated (a "military-grade" guard that is only procedural, a "zero network" tool whose source greps out a page of provider API endpoints, an OS that is second-class behind the claim). No cheerleading, no two-sided hedging to avoid a verdict.
- Say what you did NOT do ("只做了源码审查,没安装") and offer the next concrete step.
- Project-specific findings (version numbers, star counts, contributor names, dates) belong in the answer only — never persisted into this skill.
Pitfalls
- A README security table is a claim, not evidence. At minimum identify where each key
comes from and whether every read path has a gate; the full trap list is in
references/claim-verification-checklist.md. #[cfg(unix)]/chmod-style hardening is a no-op on Windows — never repeat it back as active protection.- Commit count is not health; count distinct contributors first.
- Separate "it works" from "as marketed": a tool can be genuinely fine while specific claims about it are false, and the review must say both.
- "Agent-ready" inverts easily: an agent-connector that injects skill/rule text telling the agent to run a plaintext-read command when convenient has made its own guardrail procedural — read the injected skill text and the tool's own agent examples as part of the review.
04 同仓库更多 Skills
web-retrieval-resilience
Web tools fail or search rules break: repair or fall back.
blocked-page-recovery
Use when a fetch fails: 403/429, paywall, WAF, bot wall.
verification-loop
Run a six-phase verification of a Claude Code session's work — build, type check, lint, tests with coverage, security grep, and diff review — then produce a PASS/FAIL verification report. Use when verifying work after completing a feature or refactor, before creating a PR, or when quality gates must pass.
unreal-world-builder
开放世界与环境专家——精通 UE5 World Partition、Landscape、程序化植被、HLOD 和大规模关卡流式加载,打造无缝开放世界体验
unreal-technical-artist
Unreal Engine 视觉管线专家——精通材质编辑器、Niagara 特效、程序化内容生成和 UE5 项目的美术到引擎管线
unreal-systems-engineer
性能与混合架构专家——精通 C++/Blueprint 边界、Nanite 几何体、Lumen GI 和 Gameplay Ability System,面向 AAA 级 Unreal Engine 项目