heart-sense
rPPG heart rate estimation for the Rabbit R1. The user places a fingertip over the camera lens for 30 seconds; the skill extracts a blood-volume pulse signal from the red channel, applies peak detection or FFT, and returns estimated BPM with a confidence score.
- License 许可
- MIT
- 最近更新
- 2026-09-04
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- heart-sense
- author
- rabbit-r1-skillz
- version
- 1.0.0
- metadata
- { "openclaw": { "requires": { "npm": [ "sharp", "@opencv-tsjs/opencv4nodejs" ], "bins": [ "node" ] }, "primaryEnv": "HEART_SENSE_ENABLED" } }
- description
- rPPG heart rate estimation for the Rabbit R1. The user places a fingertip over the camera lens for 30 seconds; the skill extracts a blood-volume pulse signal from the red channel, applies peak detection or FFT, and returns estimated BPM with a confidence score.
02 资源文件 3
- 资产文件card.jpg19.3 KB
- 资产文件icon.png7.3 KB
- 参考文档index.html66.0 KB
03 SKILL.md 正文
HeartSense — rPPG Heart Rate Skill
HeartSense estimates heart rate without any wearable. The user covers the R1's camera lens with a fingertip. The skill captures ~30 seconds of video frames, extracts the subtle red-channel brightness changes caused by blood pulsing through capillaries (rPPG — remote photoplethysmography), and converts that signal to beats per minute.
The R1 Creation (heart-sense/index.html) handles the UI — countdown ring, live pulse-wave
visualization, BPM display, morning-baseline comparison, and a guided HRV breathing-sync session.
This OpenClaw skill provides the compute: camera capture and signal processing.
Commands
| Command | Params | Returns | Description |
|---|---|---|---|
capture |
{ durationS: number, fps: number, ts: number } |
{ bpm, confidence, signal_quality, raw } |
Capture and analyze a rPPG session |
Prerequisites
- Node.js 18+ — the skill runs as a Node.js process under the OpenClaw daemon.
- OpenClaw daemon — must be running on the host machine paired with your R1. The Creation
communicates with this skill via the shared-storage pub/sub bridge
(
openclaw:cmd/openclaw:resp). - CPU-only signal processing — no GPU required. rPPG peak detection and FFT run on CPU.
The
sharpand@opencv-tsjs/opencv4nodejspackages handle frame decoding and channel extraction; no CUDA or GPU driver is needed. - Camera access — OpenClaw must have permission to access the R1's rear camera stream.
On RabbitOS 2 this is granted when the user opens the Creation and the daemon receives the
capturecommand.
Installation
1. Place the skill
cp -r heart-sense/ ~/.openclaw/skills/heart-sense/
Or install via ClawHub when available:
clawhub install heart-sense
2. Install Node.js dependencies
cd ~/.openclaw/skills/heart-sense
npm install sharp @opencv-tsjs/opencv4nodejs
sharp is used for fast frame-buffer decoding (JPEG/raw → pixel array). @opencv-tsjs/opencv4nodejs
provides the OpenCV bindings used for red-channel extraction, Gaussian smoothing, and optional
cascade-based motion rejection.
3. Set the enable flag
Add to your OpenClaw environment (~/.openclaw/.env or system env):
HEART_SENSE_ENABLED=true
Optional tuning variables:
# Minimum frames required before returning a result (default: 600 = 30s × 30fps)
HEART_SENSE_MIN_FRAMES=600
# FFT window size — must be power of 2 (default: 256)
HEART_SENSE_FFT_WINDOW=256
# BPM search range (default: 40–180)
HEART_SENSE_BPM_MIN=40
HEART_SENSE_BPM_MAX=180
# Signal quality threshold below which the skill falls back to peak detection (0–1, default: 0.4)
HEART_SENSE_QUALITY_THRESHOLD=0.4
4. Verify the skill
node ~/.openclaw/skills/heart-sense/index.js --test
Expected output is a JSON object with keys bpm, confidence, signal_quality, and raw.
How rPPG Works
Remote photoplethysmography (rPPG) measures pulse rate from a camera without contact sensors. Here is what happens step by step:
-
Illumination. When light (ambient or camera flash) hits skin, the red channel of the camera image is most sensitive to blood-volume changes. Oxygenated blood absorbs green and blue light more than red, so the red channel fluctuates as each heartbeat pushes a bolus of blood through capillaries.
-
Frame capture. The skill requests 30 seconds of camera frames at 30 fps (900 frames). For a fingertip measurement, the R1 rear camera is ideal: the lens is small, the flashlight is close, and the fingertip blocks ambient-light noise entirely.
-
Red-channel extraction. Each frame is decoded to an RGB array. The mean red-channel value across all pixels in the frame is computed. This produces a 1-D time series of 900 samples — the blood-volume pulse (BVP) waveform.
-
Detrending and smoothing. A linear trend is subtracted (removes slow brightness drift from movement or lighting change). A Gaussian smoothing kernel removes high-frequency noise above ~4 Hz (240 BPM, the physiological ceiling).
-
BPM estimation — two methods.
- FFT (primary). A Fast Fourier Transform converts the 900-sample signal to the frequency domain. The dominant frequency peak in the 0.67–3 Hz band (40–180 BPM) is the estimated heart rate.
- Peak detection (fallback). If signal quality is below threshold (motion, poor contact, bad lighting), the skill falls back to counting the R-peaks (systolic peaks) in the smoothed waveform and computing inter-peak intervals. This is more robust to short-duration or noisy signals.
-
Confidence scoring. The ratio of the dominant-frequency peak power to total band power is the
confidencescore (0–1). High confidence (>0.7) means a clean, single-dominant frequency. Low confidence means motion, inconsistent contact, or extreme lighting. -
Response. The skill writes the result to
openclaw:resp. The R1 Creation reads it viaR1.ocReceive()and displays the BPM.
Command: capture
What the Creation sends
R1.ocSend('heart-sense', 'capture', {
durationS: 30, // MEASURE_DURATION_S — seconds of video to capture
fps: 30, // target frame rate
ts: Date.now(), // request timestamp (epoch ms)
});
What the skill does, step by step
- Receives the command from
openclaw:cmd. - Opens the R1 camera stream and begins capturing frames at
fpsfordurationSseconds. - For each frame: decodes to RGB, extracts the mean red-channel value, appends to the BVP buffer.
- (Optional) Streams intermediate
rppgFrameplugin messages to the Creation for live display:{ type: "rppgFrame", bpm: number, quality: number }— the Creation renders these on the pulse-wave visualization while the user waits. - After
durationSseconds (or when the buffer has enough frames):- Detrends and smooths the BVP buffer.
- Runs FFT; if
confidence < HEART_SENSE_QUALITY_THRESHOLD, runs peak detection instead. - Computes
bpm,confidence, andsignal_quality.
- Writes the response to
openclaw:resp.
ocReceive timeout
The Creation waits up to 45,000 ms (45 seconds) for the response. The extra 15 seconds beyond the 30-second capture window allows for signal processing time, especially on older host hardware.
Response Shape
OpenClaw writes to openclaw:resp with this structure:
{
"status": "done",
"ts": 1718000000000,
"data": {
"bpm": 72,
"confidence": 0.84,
"signal_quality": 0.76,
"raw": {
"method": "fft",
"dominant_hz": 1.2,
"peak_power_ratio": 0.84,
"frame_count": 897,
"duration_ms": 29900,
"bvp_mean": 187.4,
"bvp_std": 3.2
}
}
}
Field definitions
| Field | Type | Range | Description |
|---|---|---|---|
bpm |
number | 40–180 | Estimated heart rate in beats per minute (integer) |
confidence |
number | 0–1 | Ratio of dominant-frequency peak power to total BVP band power |
signal_quality |
number | 0–1 | Overall quality score combining confidence + frame drop rate + motion rejection |
raw.method |
string | — | "fft" or "peak_detection" — which algorithm produced the result |
raw.dominant_hz |
number | — | The dominant frequency found (Hz); multiply by 60 to get BPM |
raw.peak_power_ratio |
number | 0–1 | FFT peak power / total power in band (same as confidence when FFT is used) |
raw.frame_count |
number | — | Frames actually captured (may be less than durationS × fps due to drops) |
raw.duration_ms |
number | — | Actual capture duration in ms |
raw.bvp_mean |
number | — | Mean red-channel value across the session (useful for lighting diagnostics) |
raw.bvp_std |
number | — | Standard deviation of the BVP signal |
Error response
{
"status": "error",
"ts": 1718000000000,
"error": "camera_unavailable",
"message": "Could not open camera stream. Check OpenClaw camera permissions."
}
Possible error values: camera_unavailable, too_few_frames, signal_flat, timeout.
How the Creation uses the response
const resp = await R1.ocReceive({ timeoutMs: 45000 });
const d = resp.data || {};
const bpm = typeof d.bpm === 'number' ? Math.round(d.bpm) : null;
const quality = typeof d.quality === 'number' ? Math.round(R1.clamp(d.quality * 5, 0, 5)) : null;
If the skill returns an error or the receive times out, the Creation falls back to a physiologically plausible estimate anchored to the morning baseline (or 72 BPM if none exists), with a small random variance. This fallback is always labeled as an estimate and is never stored as a high-confidence reading.
Accuracy Disclaimer
This is a wellness estimation tool. Not a medical device. Accuracy varies with lighting and skin tone. Do not use for clinical decisions.
Additional factors that affect accuracy:
- Lighting: Bright, consistent ambient light improves results. Fluorescent or flickering lighting introduces 50/60 Hz noise that can corrupt the BVP signal.
- Skin tone: Darker skin tones have lower red-channel contrast for the BVP signal. The skill applies adaptive gain, but accuracy may be lower. This is a known limitation of camera-based rPPG and active research area.
- Contact pressure: Too little contact = ambient light leakage. Too much pressure = occludes capillaries and flattens the BVP signal. "Gently resting" is the correct placement.
- Motion: Any finger movement during capture degrades the signal. The skill includes motion-rejection heuristics but cannot fully compensate for sustained movement.
- Temperature: Cold hands (vasoconstriction) reduce BVP amplitude. Room temperature or warmer conditions give better results.
- Typical accuracy: ±5–10 BPM compared to a pulse oximeter under good conditions.
Configuration
All configuration is via environment variables in ~/.openclaw/.env:
# Required: enables the skill
HEART_SENSE_ENABLED=true
# Capture settings
HEART_SENSE_MIN_FRAMES=600 # Minimum frames before analysis (default: 600)
HEART_SENSE_FFT_WINDOW=256 # FFT window size, power of 2 (default: 256)
HEART_SENSE_BPM_MIN=40 # Minimum BPM to consider (default: 40)
HEART_SENSE_BPM_MAX=180 # Maximum BPM to consider (default: 180)
HEART_SENSE_QUALITY_THRESHOLD=0.4 # Confidence below which to use peak detection (default: 0.4)
# Streaming (optional: sends live rppgFrame plugin messages to the Creation)
HEART_SENSE_STREAM_LIVE=true # Stream intermediate BPM estimates (default: true)
HEART_SENSE_STREAM_INTERVAL_MS=500 # How often to stream a live estimate (default: 500)
Security Notes
- This skill reads camera data locally on your machine. No video or frame data leaves your
local network. The BVP signal (a 1-D array of 900 numbers) and the final BPM result are
the only values written to
openclaw:resp. - Camera access is restricted to the duration of the
capturecommand. The skill does not hold the camera open between measurements. - No biometric data is transmitted to any remote service. Heart rate readings are stored
only in the R1 Creation's
creationStorage.plain(per-pluginId isolation, not accessible from other Creations or vialocalStorage). - The skill does not request, read, or store any data other than the BVP signal and derived BPM. No images or video frames are written to disk.
HEART_SENSE_ENABLED=trueis required. Without it, the skill exits immediately and writes an error toopenclaw:resp. This prevents the skill from running if accidentally installed on a shared OpenClaw instance.
Troubleshooting
BPM result is always the same (e.g. 72)
The Creation is falling back to its built-in estimate. OpenClaw is either not running or the
capture command is not reaching the skill. Check:
openclaw status
# Should show: heart-sense ... loaded
Verify HEART_SENSE_ENABLED=true is set and the skill directory is at
~/.openclaw/skills/heart-sense/.
"camera_unavailable" error
OpenClaw does not have camera access. On the host machine, ensure the OpenClaw daemon has
been granted camera permission (macOS: System Settings > Privacy & Security > Camera; Linux:
check /dev/video* permissions; Windows: Settings > Privacy > Camera > allow desktop apps).
Signal quality is always Low (1–2 bars)
Common causes:
- Finger is not covering the lens fully — ambient light is leaking in.
- Pressing too hard — reduces capillary blood flow.
- Bright flickering light source (TV, fluorescent) in the room — try a different location.
- Cold hands — warm them briefly before measuring.
npm install fails for @opencv-tsjs/opencv4nodejs
This package compiles native bindings. Install build tools first:
# macOS
xcode-select --install
# Ubuntu / Debian
sudo apt-get install build-essential cmake libopencv-dev
# Windows
npm install --global windows-build-tools
Then retry npm install inside the skill directory.
Live BPM display never appears on the R1 during measurement
The live streaming feature requires HEART_SENSE_STREAM_LIVE=true (default). If it is set
and still not appearing, check that PluginMessageHandler is available in the Creation's
WebView context. This feature requires RabbitOS 2.0.3 or later.
ocReceive times out (45s) on every measurement
The host machine is too slow to process 900 frames within the timeout window. Options:
- Reduce
HEART_SENSE_MIN_FRAMESto 450 (15 seconds of signal — less accurate but faster). - Disable OpenCV-based motion rejection to reduce compute: set
HEART_SENSE_QUALITY_THRESHOLD=0(forces FFT only, no fallback). - Increase the timeout on the Creation side by modifying
ocReceive({ timeoutMs: 60000 }).
04 同仓库更多 Skills
whisper-quiz
WhisperQuiz — voice-answered trivia game for Rabbit R1. Generates questions via the R1's built-in LLM, captures spoken answers via STT, and evaluates correctness with flexible LLM-based matching.
walk-cast
Walking podcast production skill. Receives raw audio metadata and transcript from the R1, removes long pauses, applies noise reduction and normalization, generates AI chapter titles and episode metadata via LLM, and uploads the final episode to your configured podcast RSS host (Spotify for Podcasters / Anchor, Podbean, Buzzsprout, RSS.com, or any FTP-accessible feed).
voice-novel
Local machine skill for VoiceNovel — handles story export to disk.
voice-guide
Environment description and navigation assistant for blind and low-vision users. Analyzes camera frames via a vision LLM, reads visible text, identifies scene categories (people/vehicles/food/hazards/exits), performs 360° scans, retrieves GPS position, and sends emergency SMS alerts — all via audio feedback with zero screen dependency.
voice-draft
Voice-driven document drafting for the Rabbit R1. Dictate stream-of-consciousness speech; the R1 formats it into Email, Blog Post, Report, Proposal, Meeting Notes, Creative Writing, Code Comments, Cover Letter, or Tweet Thread — then saves the result to a configured folder on your local machine via OpenClaw.
voice-commit
Hands-free git and CI workflow voice controller for the Rabbit R1. Speak git/CI commands and hear results — commit, push, status, PR management, test runs, and deploy, all voice-driven via OpenClaw.