[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7ARYZGCSHSQM6TJVRBX9TCJ":3,"skill-same:01M7ARYZGCSHSQM6TJVRBX9TCJ":48},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":27,"repositoryHtmlUrl":28,"repositoryHomepageUrl":29,"ownerHtmlUrl":30,"ownerBlogUrl":31,"ownerDescription":32,"updatedAt":33,"install":34,"resources":47},"01M7ARYZGCSHSQM6TJVRBX9TCJ","robotics-stack-ros2-and-perception",null,"Use when architecting a robot stack or working on its sensing layer: the layers of a real robot stack and behaviour orchestration, ROS 2 and what it actually is — the distributions, DDS and the middleware layer, and the alternatives — perception and sensor fusion across cameras, lidar, radar and depth, the eternal time-and-transform bugs, and state estimation and SLAM. Includes the router for the whole robotics-software reference.","the-vibey-project\u002Fvibey","the-vibey-project","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F329298758?v=4","01M7AJHGZ9722CTJ4JNQZVRH0Q","the vibey project","01M7AJHGZVMG2K7TGXQ6XFHVG3",5,"MIT","https:\u002F\u002Fgithub.com\u002Fthe-vibey-project\u002Fvibey\u002Fblob\u002Fdevelop\u002Fsrc\u002Fvibey_tools\u002Fskills\u002Fplugins\u002Frobotics-software\u002Fskills\u002Frobotics-stack-ros2-and-perception\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: robotics-stack-ros2-and-perception\ndescription: \"Use when architecting a robot stack or working on its sensing layer: the layers of a real robot stack and behaviour orchestration, ROS 2 and what it actually is — the distributions, DDS and the middleware layer, and the alternatives — perception and sensor fusion across cameras, lidar, radar and depth, the eternal time-and-transform bugs, and state estimation and SLAM. Includes the router for the whole robotics-software reference.\"\n---\n\n# Robotics Software: The Stack, ROS 2, Perception, and State Estimation\n\n> **Part 1 of 5** of the *Robotics Software* reference (plugin `robotics-software`), covering §0–§4. Sibling skills: `robotics-planning-control-and-manipulation` (§5–§7), `robotics-learning-simulation-and-fleets` (§8–§10), `robotics-safety-standards-and-deployment` (§11–§14), `robotics-reference` (§15–§20). Section numbers are shared across the set; a reference written as §N → `skill` points into that sibling skill.\n>\n> **Currency:** Verified August 2026. See §17 → `robotics-reference` for the currency snapshot and what goes stale first.\n\n> **How to read this.** Reference for engineers building robots that operate in the\n> physical world with real consequences — deliberately distinct from a hobbyist kit\n> reference. Three markers:\n> - **[DURABLE]** — control theory, estimation, architecture, and safety discipline.\n>   Most of this document.\n> - **[VERSIONED]** — frameworks, models, standards, tooling.\n> - **[CONTESTED]** — genuine and active disagreement, of which this field currently has\n>   a lot.\n>\n> **⚠️ GOTCHA** boxes mark the failures that damage hardware, or people.\n>\n> **The three framings that organize everything below:**\n> 1. **Physics doesn't have an undo button.** A web service that fails returns a 500;\n>    a robot that fails puts mass through space. **This single asymmetry justifies every\n>    piece of apparently excessive rigour below** — the redundancy, the state machines,\n>    the simulation, the safety cases (§11 → `robotics-safety-standards-and-deployment`, §13 → `robotics-safety-standards-and-deployment`).\n> 2. **⚠️ Robotics is an integration discipline, and the integration is where it fails.**\n>    Perception, estimation, planning, and control each work in isolation and break at\n>    the seams — timing, coordinate frames, latency, and units. **The classic robotics bug\n>    is not a bad algorithm; it's a transform published 40ms late** (§3.4, §12 → `robotics-safety-standards-and-deployment`).\n> 3. **The field is in the middle of a genuine methodological argument** between\n>    classical model-based robotics and learned end-to-end policies (§8 → `robotics-learning-simulation-and-fleets`, §16.1 → `robotics-reference`).\n>    **Neither side has won, production systems are overwhelmingly hybrid, and anyone\n>    telling you the argument is settled is selling something.**\n\n---\n\n## §0. Routing\n\n| Asked about... | Go to |\n|---|---|\n| The overall stack architecture | §1 |\n| ROS 2, middleware, and alternatives | §2 |\n| Perception and sensors | §3 |\n| State estimation, SLAM, localization | §4 |\n| Motion planning | §5 → `robotics-planning-control-and-manipulation` |\n| **Control — PID through whole-body** | **§6 → `robotics-planning-control-and-manipulation`** |\n| Manipulation and grasping | §7 → `robotics-planning-control-and-manipulation` |\n| **Learning: RL, imitation, and VLA models** | **§8 → `robotics-learning-simulation-and-fleets`** |\n| Simulation and sim-to-real | §9 → `robotics-learning-simulation-and-fleets` |\n| Multi-robot and fleets | §10 → `robotics-learning-simulation-and-fleets` |\n| Real-time and safety-critical engineering | §11 → `robotics-safety-standards-and-deployment` |\n| Testing, debugging, field deployment | §12 → `robotics-safety-standards-and-deployment` |\n| **Functional safety and standards** | **§13 → `robotics-safety-standards-and-deployment`** |\n| Aerospace flight software | §14 → `robotics-safety-standards-and-deployment` |\n| \"Don't do this\" | §15 → `robotics-reference` |\n| \"Which side is right?\" | §16 → `robotics-reference` |\n| \"Is this still current?\" | §17 → `robotics-reference` |\n| Books, courses, people | §18 → `robotics-reference` |\n\n---\n\n## §1. The Stack\n\n**[DURABLE] Every serious robot stack has these layers, whatever the vendor calls them:**\n\n```\n   MISSION \u002F TASK PLANNING     what should the robot accomplish?      (§1.2)\n            ↓\n   BEHAVIOUR \u002F EXECUTIVE        behaviour trees, state machines       (§1.2)\n            ↓\n   MOTION PLANNING              collision-free trajectories           (§5)\n            ↓\n   CONTROL                      track the trajectory. 100Hz–10kHz     (§6)\n            ↓\n   HARDWARE ABSTRACTION         drivers, EtherCAT, CAN, actuators\n   ─────────────────────────────────────────────────────────────\n   PERCEPTION → STATE ESTIMATION feeds every layer above       (§3, §4)\n   ─────────────────────────────────────────────────────────────\n   SAFETY LAYER                 ⚠️ independent, often separate silicon (§13)\n```\n\n**[DURABLE] The rate hierarchy is the thing to internalize**, because it dictates\narchitecture: **control loops run at 100Hz–10kHz with hard deadlines; state estimation at\n50–500Hz; perception at 10–60Hz; planning at 1–10Hz; mission planning whenever.**\n⚠️ **The layers must be decoupled such that a slow planner cannot stall a fast controller** —\nthat decoupling is the single most important structural decision in the stack.\n\n### 1.2 Behaviour orchestration\n**Finite state machines** — explicit, verifiable, ⚠️ **and they explode combinatorially**\nas behaviours multiply. **Behaviour trees** — ⚠️ **the modern default**, because they're\nmodular, reactive, and composable; `BehaviorTree.CPP` and Nav2's BT navigator are the\ncommon implementations. **Task and motion planning (TAMP)** integrates symbolic planning\nwith geometric feasibility. **⚠️ LLM\u002FVLM-driven task planning** is the newest layer (§8.2 → `robotics-learning-simulation-and-fleets`)\nand the least mature.\n\n**[DURABLE] Whatever you use, the requirement is the same: at any moment you must be able\nto say what the robot is doing and why, and you must be able to stop it.**\n\n---\n\n## §2. ROS 2 and Middleware\n\n### 2.1 What ROS actually is\n\n**[DURABLE] Despite the name, ROS is not an operating system** — it's an SDK plus a\nmiddleware plus a package ecosystem. **The value was never the message passing; it's that\nthousands of drivers, algorithms, and tools speak the same interfaces.**\n\n**⚠️ The ROS 1 → ROS 2 change that matters**: ROS 1 had a **central `rosmaster`** — a\nsingle point of failure with custom TCP\u002FUDP transports, no security, and no real-time\nguarantees. **ROS 2 has no master**; discovery is distributed via **DDS**, the\npublish-subscribe standard used in aerospace and defence. **ROS 1 support ended May 2025.\nNew work is ROS 2, without qualification.**\n\n### 2.2 [VERSIONED] The distributions\n\n**Release cadence: one release every year on 23 May.** **Even years are LTS with five\nyears of support; odd years get 18 months.** Each release targets exactly one Ubuntu LTS.\n\n| Distro | Status |\n|---|---|\n| **Lyrical Luth** (May 2026) | ⚠️ **Current release**, on Ubuntu 26.04. Patch releases through 2026 |\n| **Kilted Kaiju** (May 2025) | Supported to **~Nov\u002FDec 2026**. First release with **Zenoh as Tier 1 middleware** |\n| **Jazzy Jalisco** (May 2024) | ⚠️ **The LTS most production systems are on** — supported to 2029, Ubuntu 24.04 |\n| **Humble Hawksbill** (2022) | Still patched (Patch Release 14, Feb 2026); Ubuntu 22.04 |\n\n**[DURABLE] For production, target the LTS.** Jazzy today; the next LTS lands May 2028.\n\n### 2.3 The middleware layer\n\n**RMW implementations** are pluggable: **Fast DDS** (default; ⚠️ **note `fastrtps` was\nrenamed `fastdds` in Kilted**), **Cyclone DDS** (⚠️ **widely preferred for reliability and\nsimplicity**), **RTI Connext** (commercial, aerospace\u002Fdefence pedigree, safety-certified\nvariants), and **Zenoh** — ⚠️ **a genuine shift: Tier 1 from Kilted, expected to be more\nefficient and more secure than DDS, and notably better across complex network\nenvironments** where DDS multicast discovery struggles.\n\n> **⚠️ GOTCHA — the ROS 2 problems that bite in production:**\n> - **DDS discovery on large networks.** ⚠️ **Multicast discovery floods, and it's the\n>   most common \"why is my robot fleet slow\" cause.** Use discovery servers, or Zenoh.\n> - **QoS profile mismatches.** ⚠️ **Publisher and subscriber with incompatible QoS\n>   silently don't connect** — no error, just no data. Check `ros2 topic info -v` first.\n> - **Executor behaviour.** The default single-threaded executor gives you unpredictable\n>   callback scheduling. **For anything timing-sensitive, use multi-threaded or\n>   real-time executors and set thread priorities deliberately.**\n> - **ROS 2 is not real-time by itself.** §11 → `robotics-safety-standards-and-deployment`.\n> - **Serialization cost.** Use intra-process communication and zero-copy for\n>   high-bandwidth topics (camera, point cloud) or you'll burn CPU on copies.\n> - **`ament_target_dependencies()` is deprecated** in favour of modern CMake targets.\n\n### 2.4 The rest of the ecosystem and the alternatives\n**Nav2** (navigation), **MoveIt 2** (manipulation planning), **ros2_control** (hardware\nabstraction and controller lifecycle), **tf2** (⚠️ **coordinate transforms — see §3.4**),\n**rviz2**, **Foxglove** (⚠️ **the better visualization and log-inspection tool for\nserious work**), **rosbag2**, **micro-ROS** (microcontrollers).\n\n**⚠️ Not everyone uses ROS, and the reasons are legitimate.** Boston Dynamics, most\nautonomous-vehicle stacks, and aerospace flight software use bespoke internal frameworks —\nbecause they need certified real-time behaviour, tighter control over scheduling and\nmemory, or a safety case ROS can't currently support. **The alternatives worth knowing**:\n**LCM**, **Zenoh standalone**, **eProsima\u002FDDS directly**, **Cyphal\u002FUAVCAN** (⚠️ **the\nstandard for drone and spacecraft component buses**), and **PX4\u002FArduPilot** for\nflight control.\n\n---\n\n## §3. Perception\n\n**[DURABLE] Sensor modalities and their honest failure modes** — and the failure modes\nmatter more than the specs:\n\n| Sensor | Gives you | ⚠️ Fails at |\n|---|---|---|\n| **Camera (mono)** | Rich semantics, cheap | No metric scale; ⚠️ **motion blur, low light, direct sun, glass** |\n| **Stereo** | Depth by disparity | ⚠️ **Textureless surfaces — a blank wall has no disparity** |\n| **RGB-D** (structured light \u002F ToF) | Dense depth, short range | ⚠️ **Sunlight destroys structured light**; multipath on ToF |\n| **LiDAR** | Precise 3D geometry | ⚠️ **Rain, fog, dust, and retroreflectors**; sparse at range; expensive |\n| **Radar** | Velocity directly (Doppler), all-weather | Low angular resolution; ⚠️ **multipath and ghost targets** |\n| **IMU** | High-rate acceleration and angular rate | ⚠️ **Drifts. Always. It is a relative sensor** (§4) |\n| **Wheel odometry** | Cheap relative motion | ⚠️ **Slip makes it lie confidently** |\n| **F\u002FT sensors** | Contact forces | Drift, temperature sensitivity |\n| **GNSS\u002FRTK** | Global position | ⚠️ **Urban canyon, indoors, multipath, jamming\u002Fspoofing** |\n\n**[DURABLE] The design principle: every sensor fails in a specific, knowable way, and\ngood perception is chosen so that the failure modes don't correlate.** Camera + LiDAR is a\nstrong pairing because sun blinds one and rain degrades the other. **Two cameras is not\nredundancy.**\n\n**The processing stack**: detection and segmentation (⚠️ **now overwhelmingly learned, and\nthe practical constraint is latency at the edge**), 3D detection over point clouds,\ntracking (Kalman\u002Fparticle filters plus data association — ⚠️ **association is the hard\npart, not filtering**), and increasingly **BEV (bird's-eye-view) fusion** and\n**occupancy networks** as the AV-derived approach to multi-sensor fusion.\n\n### 3.4 ⚠️ Time and transforms — the two eternal bugs\n\n> **⚠️ GOTCHA — these two cause more robotics bugs than every algorithm combined.**\n>\n> **Coordinate frames.** Every measurement is in *some* frame. **`map` → `odom` → `base_link`\n> → `sensor`** is the ROS convention, and **⚠️ mixing them up produces plausible, wrong\n> behaviour rather than an error.** Use tf2 rigorously; **never hand-compose transforms**;\n> and know your conventions — ⚠️ **REP-103 says x-forward, y-left, z-up, and quaternions\n> are (x,y,z,w) in ROS and (w,x,y,z) in several libraries.** That one has cost the field\n> thousands of hours.\n>\n> **Timestamps.** ⚠️ **Sensor data is always stale by the time you use it.** Every message\n> must carry the time the measurement was *taken*, not published. Clocks across devices\n> must be synchronized (**PTP** where it matters, NTP otherwise), and **latency must be\n> budgeted end to end** — sensor → driver → processing → planning → control → actuator.\n> **A control loop acting on 200ms-old state is a control loop with 200ms of dead time,\n> and dead time destabilizes controllers** (§6 → `robotics-planning-control-and-manipulation`).\n\n---\n\n## §4. State Estimation and SLAM\n\n**[DURABLE] The core insight of the whole field: you never know where you are; you have a\nprobability distribution over where you might be.** Engineering that distribution well\n*is* state estimation.\n\n**The filter family**: **Kalman filter** (optimal for linear-Gaussian), **EKF**\n(linearize; ⚠️ **diverges when the linearization is poor**), **UKF** (sigma points,\nbetter for strong nonlinearity), **particle filter** (arbitrary distributions,\n⚠️ **suffers particle depletion**; the basis of AMCL), and **factor graphs \u002F pose graph\noptimization** — ⚠️ **which is what modern SLAM actually uses**, via GTSAM, g2o, or Ceres.\n\n**[DURABLE] The move from filtering to smoothing (factor graphs) is one of the genuine\nmethodological advances of the last two decades**: keeping and re-optimizing a window of\nhistory beats propagating a single mean and covariance forward.\n\n**SLAM in practice**: **visual** (ORB-SLAM3, and the classic), **visual-inertial**\n(VINS-Fusion, OKVIS — ⚠️ **IMU + camera is strongly complementary: the IMU covers fast\nmotion and the camera bounds drift**), **LiDAR** (LOAM family, FAST-LIO2, KISS-ICP),\n**LIO** (LiDAR-inertial), and increasingly **learned front-ends with classical back-ends**.\n\n**⚠️ The problems that actually bite**: **loop closure** (recognizing you've been here\nbefore — and ⚠️ **a false positive corrupts the whole map**), **the kidnapped robot\nproblem**, **long-term map maintenance in changing environments** (⚠️ **the underrated\none: your warehouse map is wrong the moment someone moves a pallet**), **degenerate\ngeometry** (a long featureless corridor is unobservable along its axis), and\n**scale drift** in monocular systems.\n","# Robotics Software: The Stack, ROS 2, Perception, and State Estimation\n\n> **Part 1 of 5** of the *Robotics Software* reference (plugin `robotics-software`), covering §0–§4. Sibling skills: `robotics-planning-control-and-manipulation` (§5–§7), `robotics-learning-simulation-and-fleets` (§8–§10), `robotics-safety-standards-and-deployment` (§11–§14), `robotics-reference` (§15–§20). Section numbers are shared across the set; a reference written as §N → `skill` points into that sibling skill.\n>\n> **Currency:** Verified August 2026. See §17 → `robotics-reference` for the currency snapshot and what goes stale first.\n\n> **How to read this.** Reference for engineers building robots that operate in the\n> physical world with real consequences — deliberately distinct from a hobbyist kit\n> reference. Three markers:\n> - **[DURABLE]** — control theory, estimation, architecture, and safety discipline.\n>   Most of this document.\n> - **[VERSIONED]** — frameworks, models, standards, tooling.\n> - **[CONTESTED]** — genuine and active disagreement, of which this field currently has\n>   a lot.\n>\n> **⚠️ GOTCHA** boxes mark the failures that damage hardware, or people.\n>\n> **The three framings that organize everything below:**\n> 1. **Physics doesn't have an undo button.** A web service that fails returns a 500;\n>    a robot that fails puts mass through space. **This single asymmetry justifies every\n>    piece of apparently excessive rigour below** — the redundancy, the state machines,\n>    the simulation, the safety cases (§11 → `robotics-safety-standards-and-deployment`, §13 → `robotics-safety-standards-and-deployment`).\n> 2. **⚠️ Robotics is an integration discipline, and the integration is where it fails.**\n>    Perception, estimation, planning, and control each work in isolation and break at\n>    the seams — timing, coordinate frames, latency, and units. **The classic robotics bug\n>    is not a bad algorithm; it's a transform published 40ms late** (§3.4, §12 → `robotics-safety-standards-and-deployment`).\n> 3. **The field is in the middle of a genuine methodological argument** between\n>    classical model-based robotics and learned end-to-end policies (§8 → `robotics-learning-simulation-and-fleets`, §16.1 → `robotics-reference`).\n>    **Neither side has won, production systems are overwhelmingly hybrid, and anyone\n>    telling you the argument is settled is selling something.**\n\n---\n\n## §0. Routing\n\n| Asked about... | Go to |\n|---|---|\n| The overall stack architecture | §1 |\n| ROS 2, middleware, and alternatives | §2 |\n| Perception and sensors | §3 |\n| State estimation, SLAM, localization | §4 |\n| Motion planning | §5 → `robotics-planning-control-and-manipulation` |\n| **Control — PID through whole-body** | **§6 → `robotics-planning-control-and-manipulation`** |\n| Manipulation and grasping | §7 → `robotics-planning-control-and-manipulation` |\n| **Learning: RL, imitation, and VLA models** | **§8 → `robotics-learning-simulation-and-fleets`** |\n| Simulation and sim-to-real | §9 → `robotics-learning-simulation-and-fleets` |\n| Multi-robot and fleets | §10 → `robotics-learning-simulation-and-fleets` |\n| Real-time and safety-critical engineering | §11 → `robotics-safety-standards-and-deployment` |\n| Testing, debugging, field deployment | §12 → `robotics-safety-standards-and-deployment` |\n| **Functional safety and standards** | **§13 → `robotics-safety-standards-and-deployment`** |\n| Aerospace flight software | §14 → `robotics-safety-standards-and-deployment` |\n| \"Don't do this\" | §15 → `robotics-reference` |\n| \"Which side is right?\" | §16 → `robotics-reference` |\n| \"Is this still current?\" | §17 → `robotics-reference` |\n| Books, courses, people | §18 → `robotics-reference` |\n\n---\n\n## §1. The Stack\n\n**[DURABLE] Every serious robot stack has these layers, whatever the vendor calls them:**\n\n```\n   MISSION \u002F TASK PLANNING     what should the robot accomplish?      (§1.2)\n            ↓\n   BEHAVIOUR \u002F EXECUTIVE        behaviour trees, state machines       (§1.2)\n            ↓\n   MOTION PLANNING              collision-free trajectories           (§5)\n            ↓\n   CONTROL                      track the trajectory. 100Hz–10kHz     (§6)\n            ↓\n   HARDWARE ABSTRACTION         drivers, EtherCAT, CAN, actuators\n   ─────────────────────────────────────────────────────────────\n   PERCEPTION → STATE ESTIMATION feeds every layer above       (§3, §4)\n   ─────────────────────────────────────────────────────────────\n   SAFETY LAYER                 ⚠️ independent, often separate silicon (§13)\n```\n\n**[DURABLE] The rate hierarchy is the thing to internalize**, because it dictates\narchitecture: **control loops run at 100Hz–10kHz with hard deadlines; state estimation at\n50–500Hz; perception at 10–60Hz; planning at 1–10Hz; mission planning whenever.**\n⚠️ **The layers must be decoupled such that a slow planner cannot stall a fast controller** —\nthat decoupling is the single most important structural decision in the stack.\n\n### 1.2 Behaviour orchestration\n**Finite state machines** — explicit, verifiable, ⚠️ **and they explode combinatorially**\nas behaviours multiply. **Behaviour trees** — ⚠️ **the modern default**, because they're\nmodular, reactive, and composable; `BehaviorTree.CPP` and Nav2's BT navigator are the\ncommon implementations. **Task and motion planning (TAMP)** integrates symbolic planning\nwith geometric feasibility. **⚠️ LLM\u002FVLM-driven task planning** is the newest layer (§8.2 → `robotics-learning-simulation-and-fleets`)\nand the least mature.\n\n**[DURABLE] Whatever you use, the requirement is the same: at any moment you must be able\nto say what the robot is doing and why, and you must be able to stop it.**\n\n---\n\n## §2. ROS 2 and Middleware\n\n### 2.1 What ROS actually is\n\n**[DURABLE] Despite the name, ROS is not an operating system** — it's an SDK plus a\nmiddleware plus a package ecosystem. **The value was never the message passing; it's that\nthousands of drivers, algorithms, and tools speak the same interfaces.**\n\n**⚠️ The ROS 1 → ROS 2 change that matters**: ROS 1 had a **central `rosmaster`** — a\nsingle point of failure with custom TCP\u002FUDP transports, no security, and no real-time\nguarantees. **ROS 2 has no master**; discovery is distributed via **DDS**, the\npublish-subscribe standard used in aerospace and defence. **ROS 1 support ended May 2025.\nNew work is ROS 2, without qualification.**\n\n### 2.2 [VERSIONED] The distributions\n\n**Release cadence: one release every year on 23 May.** **Even years are LTS with five\nyears of support; odd years get 18 months.** Each release targets exactly one Ubuntu LTS.\n\n| Distro | Status |\n|---|---|\n| **Lyrical Luth** (May 2026) | ⚠️ **Current release**, on Ubuntu 26.04. Patch releases through 2026 |\n| **Kilted Kaiju** (May 2025) | Supported to **~Nov\u002FDec 2026**. First release with **Zenoh as Tier 1 middleware** |\n| **Jazzy Jalisco** (May 2024) | ⚠️ **The LTS most production systems are on** — supported to 2029, Ubuntu 24.04 |\n| **Humble Hawksbill** (2022) | Still patched (Patch Release 14, Feb 2026); Ubuntu 22.04 |\n\n**[DURABLE] For production, target the LTS.** Jazzy today; the next LTS lands May 2028.\n\n### 2.3 The middleware layer\n\n**RMW implementations** are pluggable: **Fast DDS** (default; ⚠️ **note `fastrtps` was\nrenamed `fastdds` in Kilted**), **Cyclone DDS** (⚠️ **widely preferred for reliability and\nsimplicity**), **RTI Connext** (commercial, aerospace\u002Fdefence pedigree, safety-certified\nvariants), and **Zenoh** — ⚠️ **a genuine shift: Tier 1 from Kilted, expected to be more\nefficient and more secure than DDS, and notably better across complex network\nenvironments** where DDS multicast discovery struggles.\n\n> **⚠️ GOTCHA — the ROS 2 problems that bite in production:**\n> - **DDS discovery on large networks.** ⚠️ **Multicast discovery floods, and it's the\n>   most common \"why is my robot fleet slow\" cause.** Use discovery servers, or Zenoh.\n> - **QoS profile mismatches.** ⚠️ **Publisher and subscriber with incompatible QoS\n>   silently don't connect** — no error, just no data. Check `ros2 topic info -v` first.\n> - **Executor behaviour.** The default single-threaded executor gives you unpredictable\n>   callback scheduling. **For anything timing-sensitive, use multi-threaded or\n>   real-time executors and set thread priorities deliberately.**\n> - **ROS 2 is not real-time by itself.** §11 → `robotics-safety-standards-and-deployment`.\n> - **Serialization cost.** Use intra-process communication and zero-copy for\n>   high-bandwidth topics (camera, point cloud) or you'll burn CPU on copies.\n> - **`ament_target_dependencies()` is deprecated** in favour of modern CMake targets.\n\n### 2.4 The rest of the ecosystem and the alternatives\n**Nav2** (navigation), **MoveIt 2** (manipulation planning), **ros2_control** (hardware\nabstraction and controller lifecycle), **tf2** (⚠️ **coordinate transforms — see §3.4**),\n**rviz2**, **Foxglove** (⚠️ **the better visualization and log-inspection tool for\nserious work**), **rosbag2**, **micro-ROS** (microcontrollers).\n\n**⚠️ Not everyone uses ROS, and the reasons are legitimate.** Boston Dynamics, most\nautonomous-vehicle stacks, and aerospace flight software use bespoke internal frameworks —\nbecause they need certified real-time behaviour, tighter control over scheduling and\nmemory, or a safety case ROS can't currently support. **The alternatives worth knowing**:\n**LCM**, **Zenoh standalone**, **eProsima\u002FDDS directly**, **Cyphal\u002FUAVCAN** (⚠️ **the\nstandard for drone and spacecraft component buses**), and **PX4\u002FArduPilot** for\nflight control.\n\n---\n\n## §3. Perception\n\n**[DURABLE] Sensor modalities and their honest failure modes** — and the failure modes\nmatter more than the specs:\n\n| Sensor | Gives you | ⚠️ Fails at |\n|---|---|---|\n| **Camera (mono)** | Rich semantics, cheap | No metric scale; ⚠️ **motion blur, low light, direct sun, glass** |\n| **Stereo** | Depth by disparity | ⚠️ **Textureless surfaces — a blank wall has no disparity** |\n| **RGB-D** (structured light \u002F ToF) | Dense depth, short range | ⚠️ **Sunlight destroys structured light**; multipath on ToF |\n| **LiDAR** | Precise 3D geometry | ⚠️ **Rain, fog, dust, and retroreflectors**; sparse at range; expensive |\n| **Radar** | Velocity directly (Doppler), all-weather | Low angular resolution; ⚠️ **multipath and ghost targets** |\n| **IMU** | High-rate acceleration and angular rate | ⚠️ **Drifts. Always. It is a relative sensor** (§4) |\n| **Wheel odometry** | Cheap relative motion | ⚠️ **Slip makes it lie confidently** |\n| **F\u002FT sensors** | Contact forces | Drift, temperature sensitivity |\n| **GNSS\u002FRTK** | Global position | ⚠️ **Urban canyon, indoors, multipath, jamming\u002Fspoofing** |\n\n**[DURABLE] The design principle: every sensor fails in a specific, knowable way, and\ngood perception is chosen so that the failure modes don't correlate.** Camera + LiDAR is a\nstrong pairing because sun blinds one and rain degrades the other. **Two cameras is not\nredundancy.**\n\n**The processing stack**: detection and segmentation (⚠️ **now overwhelmingly learned, and\nthe practical constraint is latency at the edge**), 3D detection over point clouds,\ntracking (Kalman\u002Fparticle filters plus data association — ⚠️ **association is the hard\npart, not filtering**), and increasingly **BEV (bird's-eye-view) fusion** and\n**occupancy networks** as the AV-derived approach to multi-sensor fusion.\n\n### 3.4 ⚠️ Time and transforms — the two eternal bugs\n\n> **⚠️ GOTCHA — these two cause more robotics bugs than every algorithm combined.**\n>\n> **Coordinate frames.** Every measurement is in *some* frame. **`map` → `odom` → `base_link`\n> → `sensor`** is the ROS convention, and **⚠️ mixing them up produces plausible, wrong\n> behaviour rather than an error.** Use tf2 rigorously; **never hand-compose transforms**;\n> and know your conventions — ⚠️ **REP-103 says x-forward, y-left, z-up, and quaternions\n> are (x,y,z,w) in ROS and (w,x,y,z) in several libraries.** That one has cost the field\n> thousands of hours.\n>\n> **Timestamps.** ⚠️ **Sensor data is always stale by the time you use it.** Every message\n> must carry the time the measurement was *taken*, not published. Clocks across devices\n> must be synchronized (**PTP** where it matters, NTP otherwise), and **latency must be\n> budgeted end to end** — sensor → driver → processing → planning → control → actuator.\n> **A control loop acting on 200ms-old state is a control loop with 200ms of dead time,\n> and dead time destabilizes controllers** (§6 → `robotics-planning-control-and-manipulation`).\n\n---\n\n## §4. State Estimation and SLAM\n\n**[DURABLE] The core insight of the whole field: you never know where you are; you have a\nprobability distribution over where you might be.** Engineering that distribution well\n*is* state estimation.\n\n**The filter family**: **Kalman filter** (optimal for linear-Gaussian), **EKF**\n(linearize; ⚠️ **diverges when the linearization is poor**), **UKF** (sigma points,\nbetter for strong nonlinearity), **particle filter** (arbitrary distributions,\n⚠️ **suffers particle depletion**; the basis of AMCL), and **factor graphs \u002F pose graph\noptimization** — ⚠️ **which is what modern SLAM actually uses**, via GTSAM, g2o, or Ceres.\n\n**[DURABLE] The move from filtering to smoothing (factor graphs) is one of the genuine\nmethodological advances of the last two decades**: keeping and re-optimizing a window of\nhistory beats propagating a single mean and covariance forward.\n\n**SLAM in practice**: **visual** (ORB-SLAM3, and the classic), **visual-inertial**\n(VINS-Fusion, OKVIS — ⚠️ **IMU + camera is strongly complementary: the IMU covers fast\nmotion and the camera bounds drift**), **LiDAR** (LOAM family, FAST-LIO2, KISS-ICP),\n**LIO** (LiDAR-inertial), and increasingly **learned front-ends with classical back-ends**.\n\n**⚠️ The problems that actually bite**: **loop closure** (recognizing you've been here\nbefore — and ⚠️ **a false positive corrupts the whole map**), **the kidnapped robot\nproblem**, **long-term map maintenance in changing environments** (⚠️ **the underrated\none: your warehouse map is wrong the moment someone moves a pallet**), **degenerate\ngeometry** (a long featureless corridor is unobservable along its axis), and\n**scale drift** in monocular systems.\n",{"name":5,"description":7},"develop","ae32736a1aabddabca5d2c96d665aa0ebc7d8700","src\u002Fvibey_tools\u002Fskills\u002Fplugins\u002Frobotics-software\u002Fskills\u002Frobotics-stack-ros2-and-perception",3,"https:\u002F\u002Fgithub.com\u002Fthe-vibey-project\u002Fvibey","https:\u002F\u002Fthe-vibey-project.github.io\u002Fvibey\u002F","https:\u002F\u002Fgithub.com\u002Fthe-vibey-project","https:\u002F\u002Fvibewithadam.matthewsteinberger.com","freedom for software engineers","2026-11-15 05:47:42",{"source":8,"skillName":5,"commands":35,"prompt":45,"zipAvailable":46},[36,39,42],{"tool":37,"command":38},"npx","npx skills add the-vibey-project\u002Fvibey --skill robotics-stack-ros2-and-perception",{"tool":40,"command":41},"bunx","bunx skills add the-vibey-project\u002Fvibey --skill robotics-stack-ros2-and-perception",{"tool":43,"command":44},"pnpm","pnpm dlx skills add the-vibey-project\u002Fvibey --skill robotics-stack-ros2-and-perception","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fthe-vibey-project\u002Fvibey\u002Fblob\u002Fdevelop\u002Fsrc\u002Fvibey_tools\u002Fskills\u002Fplugins\u002Frobotics-software\u002Fskills\u002Frobotics-stack-ros2-and-perception\u002FSKILL.md",true,[],[49,56,63,70,77,84],{"skillNo":50,"name":51,"nameZh":6,"description":52,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":53,"categories":54,"keywords":55,"publishedAt":19},"01M7AT1FQJAE1FQYAMDTF490Y4","white-paper-writing","Use when writing white papers, technical reports, policy documents, thought leadership pieces, or any long-form authoritative document. Triggers on requests to write, structure, outline, or review white papers, technical reports, position papers, policy briefs, or evidence-driven arguments. Also use for B2B content marketing strategy, technical documentation requiring authority, academic-style reports, or any document that must educate while persuading. Use when someone asks \"how should I structure this\" for a long-form document.","https:\u002F\u002Fgithub.com\u002Fthe-vibey-project\u002Fvibey\u002Fblob\u002Fdevelop\u002Fsrc\u002Fvibey_tools\u002Fskills\u002Fplugins\u002Fwriting-craft\u002Fskills\u002Fwhite-paper-writing\u002FSKILL.md",[],[],{"skillNo":57,"name":58,"nameZh":6,"description":59,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":60,"categories":61,"keywords":62,"publishedAt":19},"01M7AT16BJRJJKK2MASXRZS76G","prose-technical-mechanics","Use when editing or writing prose at a technical\u002Fmechanical level: choosing between Anglo-Saxon and Latinate diction, placing information in topic vs stress positions, writing periodic vs cumulative sentences, handling sentence rhythm, punctuation as logic, transitions, paragraph structure, or phonesthetics. Use when someone wants to understand WHY prose works or doesn't, not just surface-level style advice. Triggers on word choice, sentence structure, emphasis, rhythm, diction, or any deep writing mechanics question.","https:\u002F\u002Fgithub.com\u002Fthe-vibey-project\u002Fvibey\u002Fblob\u002Fdevelop\u002Fsrc\u002Fvibey_tools\u002Fskills\u002Fplugins\u002Fwriting-craft\u002Fskills\u002Fprose-technical-mechanics\u002FSKILL.md",[],[],{"skillNo":64,"name":65,"nameZh":6,"description":66,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":67,"categories":68,"keywords":69,"publishedAt":19},"01M7AT0W9HT0ZZ9Y287NW99MR3","prose-mechanics","Use when writing, editing, or analyzing prose style; when trying to avoid generic AI-sounding writing; when applying specific prose styles such as journalistic, academic, narrative, KJV biblical, Hemingway plain, Latinate\u002FJamesian, Orwellian, Augustan, Victorian, Early Modern (Bacon, Browne, Milton), or free indirect discourse; or when coaching someone on writing voice and mechanics. Triggers on any request to improve, analyze, or write in a specific prose style, to fix \"AI-sounding\" writing, to make prose more vivid or concrete, to vary sentence rhythm, or to understand how a particular writer achieves their effects.","https:\u002F\u002Fgithub.com\u002Fthe-vibey-project\u002Fvibey\u002Fblob\u002Fdevelop\u002Fsrc\u002Fvibey_tools\u002Fskills\u002Fplugins\u002Fwriting-craft\u002Fskills\u002Fprose-mechanics\u002FSKILL.md",[],[],{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":76,"publishedAt":19},"01M7AT0KR635JX66X2X6F3E2CP","narrative-structure","Use when structuring stories, pitches, presentations, white papers, or any persuasive narrative; analyzing the hero's journey in fiction or non-fiction; applying StoryBrand for business communication; designing narrative arcs for thought leadership; or understanding Vogler, Snyder, Propp, or Campbell frameworks. Triggers on any narrative structure, storytelling, pitch structure, StoryBrand, hero's journey, three-act structure, Freytag, Save the Cat, narrative arc, or \"how should I structure this story\u002Fpitch\u002Fpresentation\" request.","https:\u002F\u002Fgithub.com\u002Fthe-vibey-project\u002Fvibey\u002Fblob\u002Fdevelop\u002Fsrc\u002Fvibey_tools\u002Fskills\u002Fplugins\u002Fwriting-craft\u002Fskills\u002Fnarrative-structure\u002FSKILL.md",[],[],{"skillNo":78,"name":79,"nameZh":6,"description":80,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":81,"categories":82,"keywords":83,"publishedAt":19},"01M7AT0AKGW09NKKZDYBT8VRDE","legal-brief-writing","Use when writing legal briefs, motions, or any court filing; when analyzing judicial cognition and persuasion; when applying elite brief-writing techniques from Roberts, Clement, Kagan, Garner; or when structuring legal arguments for maximum persuasion. Also use for autistic\u002Fneurodivergent advocates who need to understand how their natural communication style maps to elite legal advocacy. Triggers on any legal writing, brief drafting, motion preparation, or legal argument structuring request.","https:\u002F\u002Fgithub.com\u002Fthe-vibey-project\u002Fvibey\u002Fblob\u002Fdevelop\u002Fsrc\u002Fvibey_tools\u002Fskills\u002Fplugins\u002Fwriting-craft\u002Fskills\u002Flegal-brief-writing\u002FSKILL.md",[],[],{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":19},"01M7AT00VWPS4Q8B5VEMXX9NQT","journal-publishability-criteria","Use whenever a research paper, article or preprint is being written, revised, reviewed or published, and specifically before every publication of docs\u002Fpaper.md. Covers what legitimate journals actually gate on, in order of gating — the editor's desk screen (scope, novelty, package problems), what reviewers judge (methodological soundness, claims the evidence can carry, statistical and qualitative reporting), EQUATOR reporting guidelines, the ICMJE authorship criteria, COPE's view of text recycling, the near-universal generative-AI policy (no AI author, disclosure, author responsibility), TOP 2025 transparency standards, how to verify a journal is legitimate, submitting without an institution, and a pre-submission checklist. Triggers on requests to evaluate, review, prepare or submit a paper, to judge publishability, to write a declarations or data-availability section, to disclose AI use, or to check a journal.","https:\u002F\u002Fgithub.com\u002Fthe-vibey-project\u002Fvibey\u002Fblob\u002Fdevelop\u002Fsrc\u002Fvibey_tools\u002Fskills\u002Fplugins\u002Fwriting-craft\u002Fskills\u002Fjournal-publishability-criteria\u002FSKILL.md",[],[]]