AGENT SKILL

ai-sommelier

OpenClaw skill for the AI Sommelier R1 Creation. Receives camera image data for wine label OCR and analysis, processes occasion-based wine recommendations, and identifies wines from blind tasting descriptions. Routes all three command types to a vision/LLM backend on your local machine.

License 许可
MIT
最近更新
2026-09-04
Stars
★ 0
Forks
0

01 前言 Front Matter

name
ai-sommelier
author
rabbit-r1-skillz
version
1.0.0
description
OpenClaw skill for the AI Sommelier R1 Creation. Receives camera image data for wine label OCR and analysis, processes occasion-based wine recommendations, and identifies wines from blind tasting descriptions. Routes all three command types to a vision/LLM backend on your local machine.

02 资源文件 3

03 SKILL.md 正文

AI Sommelier — OpenClaw Skill

The AI Sommelier R1 Creation communicates with your local machine via the OpenClaw shared-storage bridge. This skill listens for three commands sent by the R1, routes them to a vision or text LLM backend, and writes the response back so the R1 can speak it and display results.


Commands

Command Triggered by What the skill does
scan-label PTT press in Scan mode Extracts the camera image embedded in systemPrompt, sends it to a vision LLM, returns a JSON wine analysis (identity, tasting profile, food pairings, drink window, price, alternatives)
occasion Voice submission in Occasion mode Sends the user's spoken occasion description to a text LLM, returns a JSON wine recommendation with name, meta, and spoken text
blind-taste Voice submission in Blind mode Sends the user's spoken tasting/aroma description to a text LLM, returns a JSON guess at grape variety and region with confidence and reasoning

Prerequisites

1. OpenClaw daemon

Install and run OpenClaw on the same machine that will call the LLM APIs. OpenClaw polls a shared R1 storage key for commands and writes responses back.

  • Repo: OpenClaw (community project)
  • Complete OpenClaw's own setup before configuring this skill.

2. Node.js 18+ or Python 3.10+

The skill handler runs as a local script. Examples in this document use Node.js. A Python port works equally well.

node --version   # must be >= 18

3. LLM backend — choose one

Option A — Anthropic API (recommended for scan-label)

scan-label sends a base64-encoded JPEG embedded in the system prompt, so it requires a vision-capable model. claude-sonnet-4-6 or higher is recommended.

export ANTHROPIC_API_KEY=sk-ant-...

Option B — Ollama with a vision model (free, fully local)

scan-label requires a multimodal model (e.g. llava:13b, moondream, or llava-phi3). Text-only commands (occasion, blind-taste) work with any Ollama model.

ollama pull llava:13b          # or another vision-capable model
export AI_SOMMELIER_BACKEND=ollama
export OLLAMA_HOST=http://localhost:11434
export OLLAMA_MODEL=llava:13b

Skill Installation

# Copy this skill into your OpenClaw skills directory
cp -r ai-sommelier/ ~/.openclaw/skills/ai-sommelier/

# Install Node.js dependencies
cd ~/.openclaw/skills/ai-sommelier
npm install @anthropic-ai/sdk node-fetch

Register the skill in ~/.openclaw/config.json:

{
  "skills": [
    {
      "name": "ai-sommelier",
      "path": "~/.openclaw/skills/ai-sommelier/handler.js",
      "entry": "handle",
      "commands": ["scan-label", "occasion", "blind-taste"]
    }
  ]
}

Then restart OpenClaw:

openclaw restart

How Each Command Works

scan-label

The R1 calls:

await R1.ocSend('ai-sommelier', 'scan-label', {
  systemPrompt: SCAN_SYSTEM_PROMPT,   // full system prompt; camera JPEG is embedded by the R1 SDK
  ts: Date.now(),
});
const resp = await R1.ocReceive({ timeoutMs: 60000 });

The systemPrompt field contains the expert sommelier instructions. The R1 SDK embeds the current camera frame as a base64 JPEG inside this payload via the OpenClaw camera bridge. The skill forwards the image and prompt to a vision LLM and returns a JSON wine analysis string.

Expected response shape (raw JSON string in data.text or data.result):

{
  "winery": "Château Rayas",
  "wine": "Châteauneuf-du-Pape",
  "vintage": "2019",
  "variety": "Grenache blend",
  "region": "Châteauneuf-du-Pape, Southern Rhône, France",
  "identity": "One vivid sentence introducing the wine.",
  "tasting": "Tasting profile — aromas, palate, structure, finish.",
  "food": "3-4 specific food pairing suggestions.",
  "drinkWindow": "Advice on current drinkability and aging potential.",
  "drinkStatus": "window",
  "price": "Retail $85–$120. Restaurant $160–$200.",
  "alternative": "One comparable wine at lower price and why.",
  "speakSummary": "30-60 second spoken summary for a dinner guest."
}

drinkStatus is one of: window | young | peak | past.

If no readable label is visible the LLM returns {"error": "..."} and the R1 shows an error screen.


occasion

The R1 calls:

await R1.ocSend('ai-sommelier', 'occasion', {
  userInput:    transcribedText,       // user's spoken occasion description
  systemPrompt: OCCASION_SYSTEM_PROMPT,
  ts: Date.now(),
});
const resp = await R1.ocReceive({ timeoutMs: 45000 });

The skill sends userInput as the user message and systemPrompt as the system prompt to a text LLM (vision is not needed here).

Expected response shape:

{
  "name": "Barolo by Giacomo Conterno",
  "meta": "2016–2018 vintages · Barolo DOCG · $80–$120",
  "recommendation": "3-4 sentences on why this wine fits the occasion.",
  "speakText": "Natural spoken recommendation, 20-40 seconds."
}

blind-taste

The R1 calls:

await R1.ocSend('ai-sommelier', 'blind-taste', {
  userInput:    transcribedText,       // user's spoken tasting/aroma description
  systemPrompt: BLIND_SYSTEM_PROMPT,
  ts: Date.now(),
});
const resp = await R1.ocReceive({ timeoutMs: 45000 });

The skill sends userInput and systemPrompt to a text LLM and returns a structured guess.

Expected response shape:

{
  "variety": "Pinot Noir",
  "region": "Burgundy, France",
  "confidence": "high",
  "reasoning": "2-3 sentences on what clues support the guess.",
  "speakText": "Natural spoken assessment, 20-35 seconds."
}

confidence is one of: high | medium | low.


Skill Handler (handler.js)

"use strict";
const Anthropic = require("@anthropic-ai/sdk");

const BACKEND = process.env.AI_SOMMELIER_BACKEND || "anthropic";
const OLLAMA_HOST = process.env.OLLAMA_HOST || "http://localhost:11434";
const OLLAMA_MODEL = process.env.OLLAMA_MODEL || "llava:13b";

const client = BACKEND === "anthropic" ? new Anthropic() : null;

async function handle(command, params) {
  switch (command) {
    case "scan-label":  return cmdScanLabel(params);
    case "occasion":    return cmdText(params);
    case "blind-taste": return cmdText(params);
    default:            return { error: `Unknown command: ${command}` };
  }
}

// scan-label — requires a vision-capable model
async function cmdScanLabel(params) {
  const systemPrompt = params.systemPrompt || "";
  // The R1 SDK / OpenClaw bridge extracts the JPEG from the payload.
  // By the time this handler runs, params.imageB64 holds the base64 JPEG.
  const imageB64 = params.imageB64 || "";

  if (!imageB64) {
    return { text: JSON.stringify({ error: "No camera image received. Ensure OpenClaw camera bridge is enabled." }) };
  }

  if (BACKEND === "anthropic") {
    const resp = await client.messages.create({
      model: "claude-sonnet-4-6",
      max_tokens: 1024,
      system: systemPrompt,
      messages: [{
        role: "user",
        content: [
          { type: "image", source: { type: "base64", media_type: "image/jpeg", data: imageB64 } },
          { type: "text", text: "Analyze this wine label and return the JSON as instructed." },
        ],
      }],
    });
    return { text: resp.content[0].text };
  }

  if (BACKEND === "ollama") {
    const r = await fetch(`${OLLAMA_HOST}/api/generate`, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        model: OLLAMA_MODEL,
        prompt: systemPrompt + "\n\nAnalyze this wine label and return the JSON as instructed.",
        images: [imageB64],
        stream: false,
      }),
    });
    const data = await r.json();
    return { text: data.response };
  }

  return { text: JSON.stringify({ error: "No LLM backend configured." }) };
}

// occasion + blind-taste — text only, no image
async function cmdText(params) {
  const systemPrompt = params.systemPrompt || "";
  const userInput = params.userInput || "";

  if (BACKEND === "anthropic") {
    const resp = await client.messages.create({
      model: "claude-sonnet-4-6",
      max_tokens: 512,
      system: systemPrompt,
      messages: [{ role: "user", content: userInput }],
    });
    return { text: resp.content[0].text };
  }

  if (BACKEND === "ollama") {
    const r = await fetch(`${OLLAMA_HOST}/api/generate`, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        model: OLLAMA_MODEL,
        system: systemPrompt,
        prompt: userInput,
        stream: false,
      }),
    });
    const data = await r.json();
    return { text: data.response };
  }

  return { text: JSON.stringify({ error: "No LLM backend configured." }) };
}

module.exports = { handle };

Configuration

Environment variable Required Default Description
AI_SOMMELIER_BACKEND No anthropic anthropic or ollama
ANTHROPIC_API_KEY When using Anthropic — Your Anthropic API key
OLLAMA_HOST When using Ollama http://localhost:11434 Ollama server URL
OLLAMA_MODEL When using Ollama llava:13b Model name; must be vision-capable for scan-label

Set variables in your shell profile or pass them to the OpenClaw daemon's environment:

# ~/.bashrc or ~/.zshrc
export AI_SOMMELIER_BACKEND=anthropic
export ANTHROPIC_API_KEY=sk-ant-...

Or for Ollama:

export AI_SOMMELIER_BACKEND=ollama
export OLLAMA_HOST=http://localhost:11434
export OLLAMA_MODEL=llava:13b

Response Contract

The R1 reads the response as:

const raw = resp?.data?.text || resp?.data?.result || '';

Your handler must return an object with a text or result key containing a raw JSON string (no markdown fences). The R1 calls JSON.parse() on this string after stripping any accidental backtick fences.

Timeouts:

Command R1 timeout
scan-label 60 seconds
occasion 45 seconds
blind-taste 45 seconds

If the skill does not respond within these windows the R1 shows an error toast and prompts the user to check that the skill is running.


Security

Wine label images are processed entirely on your local machine and sent only to the LLM backend you configure. No image data is written to disk by this skill. When using the Anthropic API, images are transmitted over HTTPS to Anthropic's servers and are subject to Anthropic's privacy policy. When using Ollama, all image data stays on your machine.

The accountKey used by the R1 hardware for WebSocket connections is a static bearer credential — do not expose it in logs or environment dumps. OpenClaw handles this credential; the skill handler itself never sees it.


Troubleshooting

Symptom Likely cause Fix
"No connection to OpenClaw" toast on scan OpenClaw daemon not running Run openclaw start; verify with openclaw status
scan-label returns error about no image OpenClaw camera bridge not enabled or version too old Ensure OpenClaw >= 0.4 with camera_bridge: true in openclaw.yaml
JSON parse error on R1 / blank result screen LLM returned markdown fences around the JSON The R1 strips common fence patterns; if your model wraps output differently, add a strip step in cmdScanLabel / cmdText before returning
"No LLM backend configured" in response AI_SOMMELIER_BACKEND is set to an unknown value Set it to anthropic or ollama
Anthropic API 401 / auth error Invalid or missing API key Verify ANTHROPIC_API_KEY is exported in the same shell session that runs OpenClaw
Ollama connection refused Ollama server not running Run ollama serve and confirm the model is pulled (ollama list)
scan-label times out at 60s with Ollama Vision inference too slow on this hardware Try a smaller model (moondream, llava-phi3) or increase your machine's GPU memory allocation
occasion / blind-taste return wrong structure Text-only Ollama model hallucinating JSON keys Switch to anthropic backend for these commands, or use a model known to follow JSON instructions reliably
Cellar data lost between sessions Unrelated to this skill — cellar is stored in R1 creationStorage Check R1 storage quota and whether the Creation was force-closed

04 同仓库更多 Skills

ajohnsonnow
ajohnsonnow/r1-warren

whisper-quiz

WhisperQuiz — voice-answered trivia game for Rabbit R1. Generates questions via the R1's built-in LLM, captures spoken answers via STT, and evaluates correctness with flexible LLM-based matching.

★ 0
ajohnsonnow
ajohnsonnow/r1-warren

walk-cast

Walking podcast production skill. Receives raw audio metadata and transcript from the R1, removes long pauses, applies noise reduction and normalization, generates AI chapter titles and episode metadata via LLM, and uploads the final episode to your configured podcast RSS host (Spotify for Podcasters / Anchor, Podbean, Buzzsprout, RSS.com, or any FTP-accessible feed).

★ 0
ajohnsonnow
ajohnsonnow/r1-warren

voice-novel

Local machine skill for VoiceNovel — handles story export to disk.

★ 0
ajohnsonnow
ajohnsonnow/r1-warren

voice-guide

Environment description and navigation assistant for blind and low-vision users. Analyzes camera frames via a vision LLM, reads visible text, identifies scene categories (people/vehicles/food/hazards/exits), performs 360° scans, retrieves GPS position, and sends emergency SMS alerts — all via audio feedback with zero screen dependency.

★ 0
ajohnsonnow
ajohnsonnow/r1-warren

voice-draft

Voice-driven document drafting for the Rabbit R1. Dictate stream-of-consciousness speech; the R1 formats it into Email, Blog Post, Report, Proposal, Meeting Notes, Creative Writing, Code Comments, Cover Letter, or Tweet Thread — then saves the result to a configured folder on your local machine via OpenClaw.

★ 0
ajohnsonnow
ajohnsonnow/r1-warren

voice-commit

Hands-free git and CI workflow voice controller for the Rabbit R1. Speak git/CI commands and hear results — commit, push, status, PR management, test runs, and deploy, all voice-driven via OpenClaw.

★ 0