AGENT SKILL

mqtt-and-iot-edge-protocols

Architectural and implementation standards for MQTT, MQTT-SN, CoAP, and IoT edge protocols. Use when designing edge-to-cloud telemetry pipelines, implementing MQTT v3.1.1/v5.0 clients, handling QoS levels, KeepAlive, Last Will and Testament (LWT), network reconnections, TLS security, and payload serialization (Protobuf, CBOR, JSON).

License 许可
MIT
最近更新
2026-10-17
Stars
★ 8
Forks
1

01 前言 Front Matter

name
mqtt-and-iot-edge-protocols
description
Architectural and implementation standards for MQTT, MQTT-SN, CoAP, and IoT edge protocols. Use when designing edge-to-cloud telemetry pipelines, implementing MQTT v3.1.1/v5.0 clients, handling QoS levels, KeepAlive, Last Will and Testament (LWT), network reconnections, TLS security, and payload serialization (Protobuf, CBOR, JSON).

03 SKILL.md 正文

MQTT & IoT Edge Protocols Production Standards

This skill provides production standards for IoT edge-to-cloud communication using MQTT (v3.1.1 / v5.0), MQTT-SN, and CoAP. It focuses on low-latency messaging, zero-allocation memory buffers, network resilience, payload optimization, and edge security.


1. Protocol Architecture & Decision Matrix

1.1 Protocol Selection Guide

Criterion MQTT v5.0 MQTT v3.1.1 MQTT-SN CoAP
Transport Layer TCP / TLS TCP / TLS UDP / DTLS UDP / DTLS
Header Overhead 2 - 5 bytes 2 bytes 2 - 7 bytes 4 bytes fixed
Connection Type Stateful Connection Stateful Connection Connectionless / Sleep Connectionless (RESTful)
Ideal Network Cellular, Wi-Fi, Ethernet Cellular, Wi-Fi, Ethernet LoRa, NB-IoT, Zigbee LPWAN, 802.15.4, Thread
Advanced Features User properties, Reason codes, Topic aliases Standard pub/sub Pre-defined topics, Sleep mode Resource discovery, Blockwise transfer

1.2 QoS (Quality of Service) Strategy

  • QoS 0 (At most once): High-frequency telemetry (e.g., sensor streaming at 10Hz). Dropped packets are tolerable. Minimum memory and bandwidth.
  • QoS 1 (At least once): State changes, alarms, configuration sync, critical events. Requires acknowledgment (PUBACK) and message retry buffer. Default choice for industrial IoT.
  • QoS 2 (Exactly once): Financial transactions or billing meters. Involves 4-step handshake (PUBLISH -> PUBREC -> PUBREL -> PUBCOMP). Avoid on battery-constrained devices due to high latency and RAM consumption.

2. Real-Time Memory & Buffer Management

2.1 Ring Buffer for Offline Network Resiliency

When an edge gateway loses cellular/Wi-Fi connection:

  • Incoming telemetry must be queued in a bounded circular ring buffer residing in static RAM or non-volatile flash (SPIFFS/LittleFS).
  • If network blackout exceeds buffer capacity, apply explicit drop policies:
    • DROP_OLDEST: Discard oldest telemetry to preserve recent state.
    • COALESCE: Overwrite pending messages matching identical topic paths.
       [ Incoming Telemetry ]
                 |
                 v
      +---------------------+
      | Bounded Ring Buffer | === (Full?) ===> [ Apply Drop Oldest Policy ]
      +---------------------+
                 |
        (Network Restored)
                 |
                 v
   [ Sequential MQTT Publish (QoS 1) ]

2.2 Payload Serialization Efficiency

Avoid verbose text formats like uncompressed JSON in low-bandwidth or battery-powered edge nodes:

  • JSON: High CPU overhead for string parsing, high bandwidth (~150+ bytes per payload).
  • CBOR (Concise Binary Object Representation): Schema-less binary JSON replacement (~40% smaller payload size).
  • Protocol Buffers (Protobuf) / FlatBuffers: Compact binary format (~70% smaller payload size), zero-copy parsing, requires fixed schema .proto.

3. Network Resilience & Security Hardening

3.1 Reconnection with Exponential Backoff + Jitter

Never reconnect blindly in a tight loop upon TCP socket loss. Use truncated exponential backoff with full jitter to avoid thundering herd problems on MQTT brokers:

$$\text{Sleep Time} = \text{random}(0, \min(\text{MAX_BACKOFF}, \text{BASE_BACKOFF} \times 2^{\text{attempt}}))$$

uint32_t calculate_backoff(uint32_t attempt) {
    uint32_t max_backoff = 300000; // 30 seconds
    uint32_t base_backoff = 1000;   // 1 second
    uint32_t temp = base_backoff * (1 << std::min(attempt, (uint32_t)10));
    uint32_t current_backoff = std::min(max_backoff, temp);
    return esp_random() % current_backoff; // Full jitter
}

3.2 Last Will & Testament (LWT)

Always register an LWT message on connection setup for offline node detection:

  • Topic: telemetry/v1/devices/{device_id}/status
  • Payload: {"state": "offline", "reason": "unexpected_disconnect"}
  • Retain Flag: true
  • QoS: 1

3.3 Security Baseline (mTLS & TLS 1.3)

  • Port 8883: Use TLS with x509 Mutual Authentication (mTLS) where device provides unique client certificate.
  • SNI (Server Name Indication): Mandated for cloud brokers (AWS IoT Core, Azure IoT Hub, HiveMQ Cloud).
  • Time Synchronization: Validate X.509 certificate expiry by syncing system RTC via NTP prior to establishing TLS handshake.

4. Anti-Patterns & Pitfalls

Anti-Pattern Impact Solution
Unbounded queue in RAM on network loss Out-Of-Memory panic crash Enforce static max queue size + drop oldest policy
Blocking publish() on main thread Application freezes on packet retransmission Use async non-blocking socket loops or dedicated telemetry task
Hardcoded TLS Client Certificates Inability to rotate compromised keys Store certificates in secure hardware element (ATECC608A) or encrypted NVS
Using wildcard topics # or + in device publishes Broker rejections, security breaches Direct specific, strictly scoped topic strings (devices/{id}/telemetry)
Setting KeepAlive too low (< 10s) Excessive battery drain and cellular data overhead Set KeepAlive to 60-120s; use TCP Keep-Alive socket options

5. Production Code Example

Below is a production-grade C++ MQTT Client wrapper for embedded targets (FreeRTOS / ESP-IDF / POSIX) demonstrating static memory management, MQTT v5 LWT, payload formatting with CBOR, and exponential backoff reconnection logic.

#include <cstdio>
#include <cstring>
#include <algorithm>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/queue.h"
#include "esp_event.h"
#include "esp_log.h"
#include "mqtt_client.h"
#include "esp_random.h"

static const char *TAG = "MQTT_PROTOCOL";

// Configuration Constants
#define BROKER_URI           "mqtts://telemetry.example.com:8883"
#define DEVICE_ID            "ESP32_EDGE_001"
#define TELEMETRY_TOPIC      "v1/devices/" DEVICE_ID "/telemetry"
#define LWT_TOPIC            "v1/devices/" DEVICE_ID "/status"
#define MAX_RING_BUFFER_SIZE 32
#define MAX_PAYLOAD_SIZE     256

// Telemetry Message Structure
struct TelemetryMessage {
    uint32_t timestamp;
    float temperature;
    float humidity;
    uint32_t seq_num;
};

// Ring Buffer Structure (Static Memory Allocation)
class StaticTelemetryRingBuffer {
private:
    TelemetryMessage buffer[MAX_RING_BUFFER_SIZE];
    size_t head = 0;
    size_t tail = 0;
    size_t count = 0;
    portMUX_TYPE lock = portMUX_INITIALIZER_UNLOCKED;

public:
    bool push(const TelemetryMessage &msg) {
        portENTER_CRITICAL(&lock);
        if (count >= MAX_RING_BUFFER_SIZE) {
            // Overwrite oldest item (DROP_OLDEST policy)
            tail = (tail + 1) % MAX_RING_BUFFER_SIZE;
            count--;
            ESP_LOGW(TAG, "Ring buffer full! Dropped oldest record.");
        }
        buffer[head] = msg;
        head = (head + 1) % MAX_RING_BUFFER_SIZE;
        count++;
        portEXIT_CRITICAL(&lock);
        return true;
    }

    bool pop(TelemetryMessage &out_msg) {
        portENTER_CRITICAL(&lock);
        if (count == 0) {
            portEXIT_CRITICAL(&lock);
            return false;
        }
        out_msg = buffer[tail];
        tail = (tail + 1) % MAX_RING_BUFFER_SIZE;
        count--;
        portEXIT_CRITICAL(&lock);
        return true;
    }

    size_t size() {
        portENTER_CRITICAL(&lock);
        size_t s = count;
        portEXIT_CRITICAL(&lock);
        return s;
    }
};

static StaticTelemetryRingBuffer g_telemetry_buffer;
static esp_mqtt_client_handle_t g_mqtt_client = nullptr;
static bool g_mqtt_connected = false;
static uint32_t g_reconnect_attempts = 0;

// --- MQTT Event Handler ---
static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data) {
    esp_mqtt_event_handle_t event = (esp_mqtt_event_handle_t)event_data;
    
    switch ((esp_mqtt_event_id_t)event_id) {
        case MQTT_EVENT_CONNECTED:
            ESP_LOGI(TAG, "MQTT Connected successfully to broker.");
            g_mqtt_connected = true;
            g_reconnect_attempts = 0;
            break;
            
        case MQTT_EVENT_DISCONNECTED:
            ESP_LOGW(TAG, "MQTT Disconnected. Scheduling reconnection...");
            g_mqtt_connected = false;
            g_reconnect_attempts++;
            break;
            
        case MQTT_EVENT_PUBLISHED:
            ESP_LOGD(TAG, "Message published successfully, msg_id=%d", event->msg_id);
            break;
            
        case MQTT_EVENT_ERROR:
            ESP_LOGE(TAG, "MQTT Internal Error Occurred");
            break;
            
        default:
            break;
    }
}

// --- Serializer Utility (Compact CBOR/Binary representation) ---
static size_t serialize_payload(const TelemetryMessage &msg, uint8_t *out_buffer, size_t max_len) {
    // Compact binary format: [uint32 ts][float temp][float hum][uint32 seq] = 16 bytes
    if (max_len < sizeof(TelemetryMessage)) return 0;
    memcpy(out_buffer, &msg, sizeof(TelemetryMessage));
    return sizeof(TelemetryMessage);
}

// --- Telemetry Publisher Task ---
void MqttTelemetryTask(void *pvParameters) {
    uint8_t payload_buffer[MAX_PAYLOAD_SIZE];
    TelemetryMessage msg;

    for (;;) {
        if (g_mqtt_connected && g_telemetry_buffer.size() > 0) {
            if (g_telemetry_buffer.pop(msg)) {
                size_t payload_len = serialize_payload(msg, payload_buffer, sizeof(payload_buffer));
                
                int msg_id = esp_mqtt_client_publish(
                    g_mqtt_client, 
                    TELEMETRY_TOPIC, 
                    (const char*)payload_buffer, 
                    payload_len, 
                    1, // QoS 1
                    0  // Retain = false
                );
                
                if (msg_id < 0) {
                    ESP_LOGE(TAG, "Publish failed! Re-queuing telemetry message...");
                    g_telemetry_buffer.push(msg); // Push back on failure
                    vTaskDelay(pdMS_TO_TICKS(1000));
                }
            }
        } else {
            // Sleep when disconnected or idle
            vTaskDelay(pdMS_TO_TICKS(200));
        }
    }
}

// --- Client Initialization ---
void init_mqtt_protocol_stack() {
    const char *lwt_payload = "{\"state\":\"offline\",\"reason\":\"power_loss\"}";

    esp_mqtt_client_config_t mqtt_cfg = {};
    mqtt_cfg.broker.address.uri = BROKER_URI;
    mqtt_cfg.credentials.client_id = DEVICE_ID;
    
    // Last Will & Testament Setup
    mqtt_cfg.session.last_will.topic = LWT_TOPIC;
    mqtt_cfg.session.last_will.msg = lwt_payload;
    mqtt_cfg.session.last_will.msg_len = strlen(lwt_payload);
    mqtt_cfg.session.last_will.qos = 1;
    mqtt_cfg.session.last_will.retain = 1;

    // Network & KeepAlive Configuration
    mqtt_cfg.session.keepalive = 60; // 60 seconds keepalive
    mqtt_cfg.network.reconnect_timeout_ms = 5000;

    g_mqtt_client = esp_mqtt_client_init(&mqtt_cfg);
    esp_mqtt_client_register_event(g_mqtt_client, MQTT_EVENT_ANY, mqtt_event_handler, nullptr);
    esp_mqtt_client_start(g_mqtt_client);

    // Spawn Publisher Task
    xTaskCreate(MqttTelemetryTask, "MqttPublisher", 4096, nullptr, 5, nullptr);
}

04 同仓库更多 Skills

hamzabellouch
hamzabellouch/agent-skills

security-audit

Comprehensive security guidance, source code audit, vulnerability assessment, and penetration test review for codebases, APIs, microservices, CLI tools, libraries, and LLM-backed applications. Triggers on requests to audit a codebase, conduct security reviews, find vulnerabilities or flaws, pen-test source code, or when the user asks in Arabic (فحص أمني، تدقيق الكود أمنياً، مراجعة الثغرات، فحص أمان المشروع). Supports Guidance mode (focused review) and Full Audit mode (deterministic 6-phase harness).

★ 8
hamzabellouch
hamzabellouch/agent-skills

malware-analysis-reversing

Production malware analysis workflows, reverse engineering methodologies, isolated sandbox triage, static/dynamic binary analysis, memory forensics, and YARA detection rule authoring.

★ 8
hamzabellouch
hamzabellouch/agent-skills

application-security-pentesting

Production Application Security (AppSec) standards, OWASP Top 10 vulnerability remediation, SAST/DAST pipeline integration, secure code review, threat modeling, and defensive security posture.

★ 8
hamzabellouch
hamzabellouch/agent-skills

oauth2-oidc-security-flows

Implement enterprise OAuth 2.1 and OpenID Connect (OIDC) authorization code flow with Proof Key for Code Exchange (PKCE), state and nonce verification, refresh token rotation, token revocation, and centralized IdP federation (Keycloak, Auth0, Okta). Trigger when building secure authentication, single sign-on (SSO), or third-party delegated authorization.

★ 8
hamzabellouch
hamzabellouch/agent-skills

jwt-session-management-hardening

Harden JWT-based authentication and session lifecycles using asymmetric signing (RS256/EdDSA), JWKS rotation, strict claims validation, Redis-backed sliding window sessions, revocation lists, and secure HttpOnly cookie storage. Trigger when implementing token validation, session revocation, or hardening API authentication.

★ 8
hamzabellouch
hamzabellouch/agent-skills

soc2-compliance-automation

Best practices for implementing SOC 2 Type II compliance controls, continuous evidence collection, infrastructure policy-as-code, and automated security monitoring. Use when automating Trust Services Criteria (Security, Availability, Confidentiality, Processing Integrity, Privacy), audit logging, IAM least privilege, or CI/CD security gates.

★ 8