hopper-try
Tests DeepSeek V4.1 Flash and Gemma 4 31B on the user's own prompts: finds real prompts in the project or chat, runs both side by side with time to first token, tokens per second and cost, and gives a verdict per example. Changes no code. Use when asked "there's a new DeepSeek model, how is it? test it for my use case", "DeepSeek V4.1 Flash any good? try it on my prompts", "is Gemma 4 worth trying for this", "give the new DeepSeek a vibe check", "try this prompt on a few open models", "run our real prompts through DeepSeek", "gemma 4 vs deepseek for this, which one?", "could an open model replace GPT-4.1 here", "is DeepSeek a drop-in replacement for OpenAI here", "worth switching to an open model?", "does DeepSeek handle our function calls reliably", "check if Gemma returns valid JSON for our schema" or "try Hopper". Not for testing their code, comparing files, writing prompts, running models locally (Ollama), fine-tuning or published benchmark scores; to switch the project's model, use hopper-integrate.
- License 许可
- MIT
- 最近更新
- 2026-10-28
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- hopper-try
- description
- Tests DeepSeek V4.1 Flash and Gemma 4 31B on the user's own prompts: finds real prompts in the project or chat, runs both side by side with time to first token, tokens per second and cost, and gives a verdict per example. Changes no code. Use when asked "there's a new DeepSeek model, how is it? test it for my use case", "DeepSeek V4.1 Flash any good? try it on my prompts", "is Gemma 4 worth trying for this", "give the new DeepSeek a vibe check", "try this prompt on a few open models", "run our real prompts through DeepSeek", "gemma 4 vs deepseek for this, which one?", "could an open model replace GPT-4.1 here", "is DeepSeek a drop-in replacement for OpenAI here", "worth switching to an open model?", "does DeepSeek handle our function calls reliably", "check if Gemma returns valid JSON for our schema" or "try Hopper". Not for testing their code, comparing files, writing prompts, running models locally (Ollama), fine-tuning or published benchmark scores; to switch the project's model, use hopper-integrate.
- allowed-tools
- Bash(python3 ${CLAUDE_SKILL_DIR}/scripts/hopper_try.py *)
02 资源文件 2
- 参考文档agents/openai.yaml252 B
- 脚本scripts/hopper_try.py13.8 KB
03 SKILL.md 正文
Try open models on your own prompts
Hopper serves DeepSeek V4.1 Flash (deepseek-v4.1-flash) and Gemma 4 31B (gemma-4-31b) behind an OpenAI-compatible API at https://api.withhopper.com/v1. This skill answers "would one of these work for my use case?" with the user's real prompts, not a generic demo. It measures and shows; it changes no code. Explicit instructions from the user override these steps.
How to run a model
-
The Hopper connector's
generatetool, when it's available (ChatGPT, Claude, any host with the Hopper connector). Prefer it everywhere it exists: it needs no key or network from your shell, and it returns each answer with time to first token, tokens per second, token counts and cost. Passmodels: ["deepseek-v4.1-flash", "gemma-4-31b"], the user'ssystemprompt andprompt(ormessagesfor a conversation), and keepreasoningoff unless the user asks for thinking. In ChatGPT, always usegenerate: its sandbox can't reach Hopper's API, so scripts there fail. -
This skill's script, only when there's no
generatetool and you can run commands with network access (Codex CLI, Claude Code):python3 ${CLAUDE_SKILL_DIR}/scripts/hopper_try.py --system-file prompt.txt "first example" "second example" python3 ${CLAUDE_SKILL_DIR}/scripts/hopper_try.py --examples examples.json --tools tools.json --json${CLAUDE_SKILL_DIR}is the folder holding this file (if your host doesn't fill it in, use that folder's absolute path). It runs every example on both models, streamed on a warm connection, and prints each answer with TTFT, tokens per second and token counts (it doesn't report cost; say so if the user asks, or point them to the console's usage page).--examplestakes a JSON list of strings,{"system", "prompt"}objects or{"messages": [...]};--toolssends OpenAI-format tools and prints each tool call with whether its arguments are valid JSON;--modelsnarrows the list. Standard library only. The key comes fromHOPPER_API_KEYor the project's.env; with none, the first run registers a key for the agent itself (small trial credit, no sign-up, agent registration) in~/.config/hopper/, never in the project. In a sandbox (Codex), request escalated permissions for the first run, with the reason "Try Hopper models: run your prompts on DeepSeek V4.1 Flash and Gemma 4 31B (network to withhopper.com and api.withhopper.com; writes ~/.config/hopper)", and suggest allowing commands that start withpython3and the script's path. Exit 3 (HOPPER_SANDBOX_BLOCKED) means the sandbox blocked it: rerun with escalation. -
Neither available: say Hopper can't run here and give the user
https://withhopper.com/consoleto try the models in the browser.
1. Find real prompts
- In a project: look for what the app actually sends: system prompts (
system,SYSTEM_PROMPT,prompt.txt,prompts/),messages=andchat.completions.createcalls, prompt templates, eval fixtures and test cases with sample inputs, and tool definitions. Note the model the project calls today (model=,base_url). Pick 2–5 representative inputs; fill template variables with realistic values from fixtures or tests, not placeholders. - In chat: use what the user pasted. If they gave nothing, ask for 2–5 real examples (a system prompt and a few typical user messages, or a JSON schema they need back) in one short question.
- Read the prompts; don't run the user's application.
2. Run
Run every example on both models with the same settings. Use max_tokens large enough for a full answer (1024 by default; more for long outputs). For JSON-output checks, put the schema in the prompt the way the app does and judge whether the output parses and matches. For function calls, use the script with --tools where you can; with generate (which takes no tools), put the tool definitions in the system prompt as the app describes them, ask for the call as JSON, and say this approximates native tool calling.
3. Report
For each example, a short block:
Example 1 · <what it tests, in a few words>
DeepSeek V4.1 Flash · TTFT <ms> ms · <n> tok/s · $<cost>
<answer, trimmed to the part that matters>
Gemma 4 31B · TTFT <ms> ms · <n> tok/s · $<cost>
<answer>
Verdict: <one line: which answer fits the task better and why, or both fine>
Then one line overall: which model fits this use case, and anything that failed (invalid JSON, a wrong tool, a missed instruction). If the user named their current model, say it wasn't run here unless you ran it; don't invent its numbers. Base verdicts on the answers you saw, not on reputation. Report numbers as measured; don't call either model or Hopper faster, better or cheaper than anything else.
4. Offer the next step (don't do it)
End with one question that offers to put the model in their code, and wait for the answer. In a project, offer the switch (and the key it needs); in a chat, offer the key and snippet. You may add "or test more examples first" to the same question, but never offer only more testing:
- Use it in code: an API key and the snippet. With the connector, call
create_api_key(name it after the project; pass the chosenmodel): it returns a one-time link to copy the key and a ready snippet. Never paste a full key into the chat. - Switch this project: the
hopper-integrateskill in this plugin points the project's OpenAI-compatible client at Hopper after the user says go. - Test more: more examples, a longer conversation, or other settings.
First run (setup)
When this skill is the user's first contact with Hopper and they gave no prompt, get to a live answer in under a minute:
- Ask for one short prompt from their work in a single line; if they'd rather not, use a neutral one ("Explain what an API rate limit is in two sentences.").
- Run it on both models with
generate(or the script) and show the side-by-side block. - Ask what they want to use it for: test it on their task (continue this skill with real examples), add it to their app (
create_api_keyand the snippet, orhopper-integratein a project), or audio (thehopper-speakandhopper-transcribeskills: Qwen 3 TTS and Nemotron ASR).
04 同仓库更多 Skills
hopper-transcribe
Transcribe speech in an audio or video file with Hopper speech-to-text (Nemotron ASR), with word timestamps. Takes WAV, MP3, M4A, voice memos and video such as MP4 or MOV (decoded with ffmpeg). Needs no setup: the first use registers a key for the agent itself. Use when the user says "transcribe this", "what does this recording say", "check my video", "what's said in this video", "review the voiceover in my launch video", "captions for my video", "subtitles", "caption this", "meeting notes from this recording", "summarize this call", "pull quotes from this interview", "speech to text" or "STT". Works from the spoken words only, not the visuals. Not for generating audio (hopper-speak) or running an LLM (hopper-try).
hopper-speak
Turn text into speech with Hopper text-to-speech (Qwen 3 TTS): play it aloud, save a WAV file, or preview and pick a voice. Needs no setup: the first use registers a key for the agent itself. Use when the user says "read this aloud", "say this out loud", "generate a voiceover", "voiceover for my launch video", "audio for my launch video", "narrate this script", "narration for my demo video", "make a podcast intro", "record an announcement", "voice for my ad", "make an audio file of…", "text to speech", "TTS" or "which voice should I use". Makes the speech track only, not video, music or sound effects. Not for writing a script (plain text), transcribing audio (hopper-transcribe) or running an LLM (hopper-try).
hopper-integrate
Points a project's LLM calls at a Hopper model (DeepSeek V4.1 Flash or Gemma 4 31B) through Hopper's OpenAI-compatible API: OpenAI SDK (Python or JavaScript), LangChain, Vercel AI SDK, LiteLLM, raw HTTP, LiveKit Agents, Pipecat or Vapi. Gets the project a key, shows the planned change (and, for a voice agent, time to first token on its own prompt), and edits code only after the user says go. Use when the user says "switch this app to DeepSeek on Hopper", "swap the LLM for Gemma 4", "move our OpenAI calls to Hopper", "point base_url at Hopper", "set up Hopper in this project", "add a Hopper API key to this repo" or "go ahead" after a hopper-try comparison. Not for trying or comparing models before deciding (hopper-try), speech or transcription (hopper-speak, hopper-transcribe), or running models locally.
hopper-announce
Change how Claude Code announces finished tasks out loud (the hopper-announce plugin): turn announcements off or on, pick another voice, make them shorter, or announce only tasks that take longer. Use when the user says "stop announcing", "be quiet when you finish", "use a different voice for announcements", "only tell me about long tasks" or "turn spoken updates back on".
hopper-try
Tests DeepSeek V4.1 Flash and Gemma 4 31B on the user's own prompts: finds real prompts in the project or chat, runs both side by side with time to first token, tokens per second and cost, and gives a verdict per example. Changes no code. Use when asked "there's a new DeepSeek model, how is it? test it for my use case", "DeepSeek V4.1 Flash any good? try it on my prompts", "is Gemma 4 worth trying for this", "give the new DeepSeek a vibe check", "try this prompt on a few open models", "run our real prompts through DeepSeek", "gemma 4 vs deepseek for this, which one?", "could an open model replace GPT-4.1 here", "is DeepSeek a drop-in replacement for OpenAI here", "worth switching to an open model?", "does DeepSeek handle our function calls reliably", "check if Gemma returns valid JSON for our schema" or "try Hopper". Not for testing their code, comparing files, writing prompts, running models locally (Ollama), fine-tuning or published benchmark scores; to switch the project's model, use hopper-integrate.
hopper-transcribe
Transcribe speech in an audio or video file with Hopper speech-to-text (Nemotron ASR), with word timestamps. Takes WAV, MP3, M4A, voice memos and video such as MP4 or MOV (decoded with ffmpeg). Needs no setup: the first use registers a key for the agent itself. Use when the user says "transcribe this", "what does this recording say", "check my video", "what's said in this video", "review the voiceover in my launch video", "captions for my video", "subtitles", "caption this", "meeting notes from this recording", "summarize this call", "pull quotes from this interview", "speech to text" or "STT". Works from the spoken words only, not the visuals. Not for running an LLM (hopper-try).