[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M732HTYEE8H1HB8XHT22VZ77":3,"skill-same:01M732HTYEE8H1HB8XHT22VZ77":67},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":46,"commitSha":47,"skillDirectory":48,"forksCount":14,"repositoryHtmlUrl":49,"repositoryHomepageUrl":6,"ownerHtmlUrl":50,"ownerBlogUrl":6,"ownerDescription":51,"updatedAt":52,"install":53,"resources":66},"01M732HTYEE8H1HB8XHT22VZ77","golang-observability-opentelemetry",null,"Instrumenting Go applications with OpenTelemetry for distributed tracing, Prometheus for metrics, and structured logging with slog","julio211916\u002FTlantiCAD-Studio-V0.1-alpha","julio211916","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F128201165?v=4","01M730ENY015MHFY82VKR6AGGR","José Carlos amador garcia","01M730ENYT83EWH6EXRHF9FTXV",0,"MIT","https:\u002F\u002Fgithub.com\u002Fjulio211916\u002FTlantiCAD-Studio-V0.1-alpha\u002Fblob\u002Fmain\u002Fskills\u002Fju\u002Fskills\u002Fgolang-observability-opentelemetry\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: golang-observability-opentelemetry\ndescription: \"Instrumenting Go applications with OpenTelemetry for distributed tracing, Prometheus for metrics, and structured logging with slog\"\nversion: 1.0.0\ncategory: toolchain\nauthor: Claude MPM Team\nlicense: MIT\nprogressive_disclosure:\n  entry_point:\n    summary: \"Master Go observability through OpenTelemetry distributed tracing, Prometheus metrics, and slog structured logging for production-ready microservices\"\n    when_to_use: \"Instrumenting microservices for observability, setting up distributed tracing across services, creating operational dashboards, debugging production issues, performance monitoring and optimization\"\n    quick_start: \"1. Initialize OpenTelemetry tracer and exporter 2. Add Prometheus metrics endpoint 3. Configure slog for structured logging 4. Instrument HTTP middleware 5. Implement health checks\"\n  token_estimate:\n    entry: 150\n    full: 5000\ncontext_limit: 700\ntags:\n  - observability\n  - golang\n  - opentelemetry\n  - prometheus\n  - tracing\n  - metrics\n  - logging\n  - slog\nrequires_tools: []\n---\n\n# Go Observability with OpenTelemetry\n\n## Overview\n\nModern Go applications require comprehensive observability through the three pillars: traces, metrics, and logs. OpenTelemetry provides vendor-neutral instrumentation for distributed tracing, Prometheus offers powerful metrics collection, and Go's slog package (1.21+) delivers structured logging with minimal overhead.\n\n**Key Features:**\n- 🔍 **OpenTelemetry**: Distributed tracing with context propagation\n- 📊 **Prometheus**: Metrics collection with \u002Fmetrics endpoint\n- 📝 **Structured Logging**: slog with JSON formatting and correlation IDs\n- 🎯 **Auto-Instrumentation**: HTTP\u002FgRPC middleware patterns\n- 💚 **Health Checks**: Kubernetes-ready readiness\u002Fliveness probes\n- 🔄 **Graceful Shutdown**: Clean exporter shutdown and signal handling\n\n## When to Use This Skill\n\nActivate this skill when:\n- Instrumenting microservices for production observability\n- Setting up distributed tracing across service boundaries\n- Creating operational dashboards with Prometheus\u002FGrafana\n- Debugging production performance issues or bottlenecks\n- Implementing SLOs and monitoring SLIs\n- Adding observability to existing Go applications\n- Correlating logs, traces, and metrics for debugging\n\n## Core Observability Principles\n\n### The Three Pillars\n\n1. **Traces**: Understand request flow across distributed systems\n2. **Metrics**: Measure system behavior and performance over time\n3. **Logs**: Record discrete events for debugging and audit\n\n### Correlation Strategy\n\nAll three pillars must share common identifiers:\n- **Trace ID**: Links all operations in a request\n- **Span ID**: Identifies specific operation within trace\n- **Request ID**: Correlates logs with traces and metrics\n\n## OpenTelemetry Integration\n\n### Installation\n\n```bash\ngo get go.opentelemetry.io\u002Fotel\ngo get go.opentelemetry.io\u002Fotel\u002Fsdk\ngo get go.opentelemetry.io\u002Fotel\u002Fexporters\u002Fjaeger\ngo get go.opentelemetry.io\u002Fcontrib\u002Finstrumentation\u002Fnet\u002Fhttp\u002Fotelhttp\n```\n\n### Basic Setup\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"log\"\n\n    \"go.opentelemetry.io\u002Fotel\"\n    \"go.opentelemetry.io\u002Fotel\u002Fexporters\u002Fjaeger\"\n    \"go.opentelemetry.io\u002Fotel\u002Fsdk\u002Fresource\"\n    sdktrace \"go.opentelemetry.io\u002Fotel\u002Fsdk\u002Ftrace\"\n    semconv \"go.opentelemetry.io\u002Fotel\u002Fsemconv\u002Fv1.21.0\"\n)\n\nfunc initTracer(serviceName string) (*sdktrace.TracerProvider, error) {\n    \u002F\u002F Create Jaeger exporter\n    exporter, err := jaeger.New(jaeger.WithCollectorEndpoint(\n        jaeger.WithEndpoint(\"http:\u002F\u002Flocalhost:14268\u002Fapi\u002Ftraces\"),\n    ))\n    if err != nil {\n        return nil, err\n    }\n\n    \u002F\u002F Create resource with service name\n    res, err := resource.Merge(\n        resource.Default(),\n        resource.NewWithAttributes(\n            semconv.SchemaURL,\n            semconv.ServiceName(serviceName),\n            semconv.ServiceVersion(\"1.0.0\"),\n        ),\n    )\n    if err != nil {\n        return nil, err\n    }\n\n    \u002F\u002F Create tracer provider\n    tp := sdktrace.NewTracerProvider(\n        sdktrace.WithBatcher(exporter),\n        sdktrace.WithResource(res),\n        sdktrace.WithSampler(sdktrace.AlwaysSample()), \u002F\u002F Use probability sampler in production\n    )\n\n    otel.SetTracerProvider(tp)\n    return tp, nil\n}\n\nfunc main() {\n    tp, err := initTracer(\"order-service\")\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer func() {\n        if err := tp.Shutdown(context.Background()); err != nil {\n            log.Printf(\"Error shutting down tracer: %v\", err)\n        }\n    }()\n\n    \u002F\u002F Application code...\n}\n```\n\n### Creating Spans\n\n```go\nimport (\n    \"context\"\n\n    \"go.opentelemetry.io\u002Fotel\"\n    \"go.opentelemetry.io\u002Fotel\u002Fattribute\"\n    \"go.opentelemetry.io\u002Fotel\u002Fcodes\"\n    \"go.opentelemetry.io\u002Fotel\u002Ftrace\"\n)\n\nfunc ProcessOrder(ctx context.Context, order Order) error {\n    tracer := otel.Tracer(\"order-service\")\n    ctx, span := tracer.Start(ctx, \"ProcessOrder\")\n    defer span.End()\n\n    \u002F\u002F Add attributes\n    span.SetAttributes(\n        attribute.String(\"order.id\", order.ID),\n        attribute.Int(\"order.items\", len(order.Items)),\n        attribute.Float64(\"order.total\", order.Total),\n    )\n\n    \u002F\u002F Validate order (creates child span)\n    if err := validateOrder(ctx, order); err != nil {\n        span.RecordError(err)\n        span.SetStatus(codes.Error, \"validation failed\")\n        return err\n    }\n\n    \u002F\u002F Fulfill order\n    if err := fulfillOrder(ctx, order); err != nil {\n        span.RecordError(err)\n        span.SetStatus(codes.Error, \"fulfillment failed\")\n        return err\n    }\n\n    span.SetStatus(codes.Ok, \"order processed successfully\")\n    return nil\n}\n\nfunc validateOrder(ctx context.Context, order Order) error {\n    _, span := otel.Tracer(\"order-service\").Start(ctx, \"validateOrder\")\n    defer span.End()\n\n    \u002F\u002F Validation logic...\n    return nil\n}\n```\n\n### HTTP Middleware Instrumentation\n\n```go\nimport (\n    \"net\u002Fhttp\"\n\n    \"go.opentelemetry.io\u002Fcontrib\u002Finstrumentation\u002Fnet\u002Fhttp\u002Fotelhttp\"\n)\n\nfunc main() {\n    \u002F\u002F Wrap handler with automatic tracing\n    handler := http.HandlerFunc(orderHandler)\n    wrappedHandler := otelhttp.NewHandler(handler, \"order-handler\")\n\n    http.Handle(\"\u002Forders\", wrappedHandler)\n    http.ListenAndServe(\":8080\", nil)\n}\n\n\u002F\u002F Manual instrumentation for more control\nfunc orderHandler(w http.ResponseWriter, r *http.Request) {\n    ctx := r.Context()\n    tracer := otel.Tracer(\"order-service\")\n\n    ctx, span := tracer.Start(ctx, \"orderHandler\")\n    defer span.End()\n\n    \u002F\u002F Extract order ID from request\n    orderID := r.URL.Query().Get(\"id\")\n    span.SetAttributes(attribute.String(\"order.id\", orderID))\n\n    \u002F\u002F Process order with propagated context\n    order, err := fetchOrder(ctx, orderID)\n    if err != nil {\n        span.RecordError(err)\n        http.Error(w, \"Order not found\", http.StatusNotFound)\n        return\n    }\n\n    \u002F\u002F ... handle response\n}\n```\n\n## Prometheus Metrics\n\n### Installation\n\n```bash\ngo get github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\ngo get github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\u002Fpromhttp\n```\n\n### Metric Types and Patterns\n\n```go\npackage metrics\n\nimport (\n    \"github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\"\n    \"github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\u002Fpromauto\"\n)\n\nvar (\n    \u002F\u002F Counter: Monotonically increasing value\n    httpRequestsTotal = promauto.NewCounterVec(\n        prometheus.CounterOpts{\n            Name: \"http_requests_total\",\n            Help: \"Total number of HTTP requests\",\n        },\n        []string{\"method\", \"path\", \"status\"},\n    )\n\n    \u002F\u002F Gauge: Value that can go up or down\n    activeConnections = promauto.NewGauge(\n        prometheus.GaugeOpts{\n            Name: \"active_connections\",\n            Help: \"Number of active connections\",\n        },\n    )\n\n    \u002F\u002F Histogram: Observations bucketed by value\n    httpRequestDuration = promauto.NewHistogramVec(\n        prometheus.HistogramOpts{\n            Name:    \"http_request_duration_seconds\",\n            Help:    \"HTTP request duration in seconds\",\n            Buckets: prometheus.DefBuckets, \u002F\u002F [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]\n        },\n        []string{\"method\", \"path\"},\n    )\n\n    \u002F\u002F Summary: Similar to histogram but calculates quantiles\n    dbQueryDuration = promauto.NewSummaryVec(\n        prometheus.SummaryOpts{\n            Name:       \"db_query_duration_seconds\",\n            Help:       \"Database query duration\",\n            Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001},\n        },\n        []string{\"query_type\"},\n    )\n)\n```\n\n### Metrics Middleware\n\n```go\nimport (\n    \"net\u002Fhttp\"\n    \"strconv\"\n    \"time\"\n\n    \"github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\u002Fpromhttp\"\n)\n\n\u002F\u002F Metrics middleware that instruments all HTTP handlers\nfunc MetricsMiddleware(next http.Handler) http.Handler {\n    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n        start := time.Now()\n\n        \u002F\u002F Track active connections\n        activeConnections.Inc()\n        defer activeConnections.Dec()\n\n        \u002F\u002F Wrap response writer to capture status code\n        rw := &responseWriter{ResponseWriter: w, statusCode: http.StatusOK}\n\n        \u002F\u002F Call next handler\n        next.ServeHTTP(rw, r)\n\n        \u002F\u002F Record metrics\n        duration := time.Since(start).Seconds()\n        httpRequestDuration.WithLabelValues(r.Method, r.URL.Path).Observe(duration)\n        httpRequestsTotal.WithLabelValues(r.Method, r.URL.Path, strconv.Itoa(rw.statusCode)).Inc()\n    })\n}\n\ntype responseWriter struct {\n    http.ResponseWriter\n    statusCode int\n}\n\nfunc (rw *responseWriter) WriteHeader(code int) {\n    rw.statusCode = code\n    rw.ResponseWriter.WriteHeader(code)\n}\n\n\u002F\u002F Expose metrics endpoint\nfunc main() {\n    http.Handle(\"\u002Fmetrics\", promhttp.Handler())\n\n    handler := MetricsMiddleware(http.HandlerFunc(orderHandler))\n    http.Handle(\"\u002Forders\", handler)\n\n    http.ListenAndServe(\":8080\", nil)\n}\n```\n\n### Custom Metrics Example\n\n```go\nfunc ProcessPayment(ctx context.Context, payment Payment) error {\n    timer := prometheus.NewTimer(dbQueryDuration.WithLabelValues(\"payment_insert\"))\n    defer timer.ObserveDuration()\n\n    \u002F\u002F Process payment\n    if err := db.InsertPayment(payment); err != nil {\n        httpRequestsTotal.WithLabelValues(\"POST\", \"\u002Fpayments\", \"500\").Inc()\n        return err\n    }\n\n    httpRequestsTotal.WithLabelValues(\"POST\", \"\u002Fpayments\", \"200\").Inc()\n    return nil\n}\n```\n\n## Structured Logging with slog\n\n### Basic Setup (Go 1.21+)\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"log\u002Fslog\"\n    \"os\"\n)\n\nfunc initLogger() *slog.Logger {\n    \u002F\u002F JSON logger for production\n    handler := slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{\n        Level: slog.LevelInfo,\n        AddSource: true, \u002F\u002F Include file:line information\n    })\n\n    logger := slog.New(handler)\n    slog.SetDefault(logger) \u002F\u002F Set as default logger\n    return logger\n}\n\nfunc main() {\n    logger := initLogger()\n\n    logger.Info(\"service starting\",\n        \"service\", \"order-service\",\n        \"version\", \"1.0.0\",\n        \"port\", 8080,\n    )\n}\n```\n\n### Context-Aware Logging\n\n```go\nimport (\n    \"context\"\n    \"log\u002Fslog\"\n\n    \"go.opentelemetry.io\u002Fotel\u002Ftrace\"\n)\n\n\u002F\u002F Add trace context to logger\nfunc LoggerWithTrace(ctx context.Context) *slog.Logger {\n    span := trace.SpanFromContext(ctx)\n    spanCtx := span.SpanContext()\n\n    return slog.With(\n        \"trace_id\", spanCtx.TraceID().String(),\n        \"span_id\", spanCtx.SpanID().String(),\n    )\n}\n\nfunc HandleRequest(ctx context.Context, req Request) error {\n    logger := LoggerWithTrace(ctx)\n\n    logger.Info(\"processing request\",\n        \"request_id\", req.ID,\n        \"method\", req.Method,\n        \"path\", req.Path,\n    )\n\n    if err := processRequest(ctx, req); err != nil {\n        logger.Error(\"request failed\",\n            \"error\", err,\n            \"duration_ms\", time.Since(req.StartTime).Milliseconds(),\n        )\n        return err\n    }\n\n    logger.Info(\"request completed successfully\",\n        \"duration_ms\", time.Since(req.StartTime).Milliseconds(),\n    )\n    return nil\n}\n```\n\n### Log Levels and Structured Fields\n\n```go\nfunc ProcessOrder(ctx context.Context, order Order) error {\n    logger := LoggerWithTrace(ctx).With(\n        \"order_id\", order.ID,\n        \"user_id\", order.UserID,\n    )\n\n    logger.Debug(\"validating order\", \"items\", len(order.Items))\n\n    if len(order.Items) == 0 {\n        logger.Warn(\"empty order received\")\n        return ErrEmptyOrder\n    }\n\n    logger.Info(\"order validation passed\")\n\n    if err := fulfillOrder(ctx, order); err != nil {\n        logger.Error(\"fulfillment failed\",\n            \"error\", err,\n            slog.Group(\"order_details\",\n                \"total\", order.Total,\n                \"items\", len(order.Items),\n            ),\n        )\n        return err\n    }\n\n    logger.Info(\"order processed successfully\",\n        \"total\", order.Total,\n    )\n    return nil\n}\n```\n\n## Health Checks and Graceful Shutdown\n\n### Health Check Endpoints\n\n```go\nimport (\n    \"context\"\n    \"database\u002Fsql\"\n    \"encoding\u002Fjson\"\n    \"net\u002Fhttp\"\n    \"time\"\n)\n\ntype HealthChecker struct {\n    db *sql.DB\n    \u002F\u002F Add other dependencies\n}\n\ntype HealthStatus struct {\n    Status      string            `json:\"status\"`\n    Version     string            `json:\"version\"`\n    Checks      map[string]string `json:\"checks\"`\n    Timestamp   time.Time         `json:\"timestamp\"`\n}\n\n\u002F\u002F Liveness probe - is the app running?\nfunc (hc *HealthChecker) LivenessHandler(w http.ResponseWriter, r *http.Request) {\n    w.Header().Set(\"Content-Type\", \"application\u002Fjson\")\n    w.WriteHeader(http.StatusOK)\n    json.NewEncoder(w).Encode(map[string]string{\n        \"status\": \"alive\",\n    })\n}\n\n\u002F\u002F Readiness probe - is the app ready to serve traffic?\nfunc (hc *HealthChecker) ReadinessHandler(w http.ResponseWriter, r *http.Request) {\n    ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)\n    defer cancel()\n\n    status := HealthStatus{\n        Status:    \"ready\",\n        Version:   \"1.0.0\",\n        Checks:    make(map[string]string),\n        Timestamp: time.Now(),\n    }\n\n    \u002F\u002F Check database\n    if err := hc.db.PingContext(ctx); err != nil {\n        status.Status = \"not_ready\"\n        status.Checks[\"database\"] = \"unhealthy: \" + err.Error()\n        w.WriteHeader(http.StatusServiceUnavailable)\n    } else {\n        status.Checks[\"database\"] = \"healthy\"\n    }\n\n    \u002F\u002F Add more dependency checks (Redis, external APIs, etc.)\n\n    w.Header().Set(\"Content-Type\", \"application\u002Fjson\")\n    if status.Status == \"ready\" {\n        w.WriteHeader(http.StatusOK)\n    }\n    json.NewEncoder(w).Encode(status)\n}\n```\n\n### Graceful Shutdown\n\n```go\nimport (\n    \"context\"\n    \"net\u002Fhttp\"\n    \"os\"\n    \"os\u002Fsignal\"\n    \"syscall\"\n    \"time\"\n)\n\nfunc main() {\n    \u002F\u002F Initialize tracer\n    tp, err := initTracer(\"order-service\")\n    if err != nil {\n        log.Fatal(err)\n    }\n\n    \u002F\u002F Setup HTTP server\n    server := &http.Server{\n        Addr:    \":8080\",\n        Handler: setupRoutes(),\n    }\n\n    \u002F\u002F Channel for shutdown signals\n    shutdown := make(chan os.Signal, 1)\n    signal.Notify(shutdown, os.Interrupt, syscall.SIGTERM)\n\n    \u002F\u002F Start server in goroutine\n    go func() {\n        slog.Info(\"server starting\", \"port\", 8080)\n        if err := server.ListenAndServe(); err != http.ErrServerClosed {\n            log.Fatal(err)\n        }\n    }()\n\n    \u002F\u002F Wait for shutdown signal\n    \u003C-shutdown\n    slog.Info(\"shutdown signal received\")\n\n    \u002F\u002F Create shutdown context with timeout\n    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)\n    defer cancel()\n\n    \u002F\u002F Shutdown HTTP server\n    slog.Info(\"shutting down HTTP server\")\n    if err := server.Shutdown(ctx); err != nil {\n        slog.Error(\"HTTP server shutdown error\", \"error\", err)\n    }\n\n    \u002F\u002F Shutdown tracer provider (flush spans)\n    slog.Info(\"shutting down tracer\")\n    if err := tp.Shutdown(ctx); err != nil {\n        slog.Error(\"tracer shutdown error\", \"error\", err)\n    }\n\n    slog.Info(\"shutdown complete\")\n}\n```\n\n## Complete Instrumentation Example\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"database\u002Fsql\"\n    \"log\u002Fslog\"\n    \"net\u002Fhttp\"\n    \"os\"\n    \"time\"\n\n    \"github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\u002Fpromhttp\"\n    \"go.opentelemetry.io\u002Fcontrib\u002Finstrumentation\u002Fnet\u002Fhttp\u002Fotelhttp\"\n    \"go.opentelemetry.io\u002Fotel\"\n    \"go.opentelemetry.io\u002Fotel\u002Fattribute\"\n)\n\ntype Server struct {\n    db     *sql.DB\n    logger *slog.Logger\n}\n\nfunc (s *Server) orderHandler(w http.ResponseWriter, r *http.Request) {\n    ctx := r.Context()\n\n    \u002F\u002F Get tracer and create span\n    tracer := otel.Tracer(\"order-service\")\n    ctx, span := tracer.Start(ctx, \"orderHandler\")\n    defer span.End()\n\n    \u002F\u002F Create context-aware logger with trace ID\n    logger := s.logger.With(\n        \"trace_id\", span.SpanContext().TraceID().String(),\n        \"request_id\", r.Header.Get(\"X-Request-ID\"),\n    )\n\n    orderID := r.URL.Query().Get(\"id\")\n    span.SetAttributes(attribute.String(\"order.id\", orderID))\n\n    logger.Info(\"fetching order\", \"order_id\", orderID)\n\n    \u002F\u002F Fetch order from database\n    order, err := s.fetchOrder(ctx, orderID)\n    if err != nil {\n        span.RecordError(err)\n        logger.Error(\"failed to fetch order\", \"error\", err)\n        http.Error(w, \"Order not found\", http.StatusNotFound)\n        return\n    }\n\n    logger.Info(\"order fetched successfully\",\n        \"order_id\", orderID,\n        \"items\", len(order.Items),\n    )\n\n    \u002F\u002F Return order as JSON\n    w.Header().Set(\"Content-Type\", \"application\u002Fjson\")\n    json.NewEncoder(w).Encode(order)\n}\n\nfunc (s *Server) fetchOrder(ctx context.Context, orderID string) (*Order, error) {\n    _, span := otel.Tracer(\"order-service\").Start(ctx, \"fetchOrder\")\n    defer span.End()\n\n    \u002F\u002F Time database query\n    start := time.Now()\n\n    var order Order\n    err := s.db.QueryRowContext(ctx, \"SELECT * FROM orders WHERE id = ?\", orderID).Scan(&order)\n\n    duration := time.Since(start).Seconds()\n    dbQueryDuration.WithLabelValues(\"select_order\").Observe(duration)\n\n    return &order, err\n}\n\nfunc setupRoutes(s *Server, hc *HealthChecker) http.Handler {\n    mux := http.NewServeMux()\n\n    \u002F\u002F Health endpoints (no tracing needed)\n    mux.HandleFunc(\"\u002Fhealth\", hc.LivenessHandler)\n    mux.HandleFunc(\"\u002Fready\", hc.ReadinessHandler)\n    mux.Handle(\"\u002Fmetrics\", promhttp.Handler())\n\n    \u002F\u002F Business endpoints (with tracing)\n    orderHandler := http.HandlerFunc(s.orderHandler)\n    mux.Handle(\"\u002Forders\", otelhttp.NewHandler(orderHandler, \"orders\"))\n\n    \u002F\u002F Wrap everything with metrics middleware\n    return MetricsMiddleware(mux)\n}\n```\n\n## Decision Trees\n\n### When to Use OpenTelemetry\n\n**Use OpenTelemetry When:**\n- Building distributed systems with multiple services\n- Need to trace requests across service boundaries\n- Debugging performance issues in microservices\n- Want vendor-neutral observability (switch backends easily)\n- Require correlation between traces, metrics, and logs\n\n**Don't Use OpenTelemetry When:**\n- Building simple monolithic applications\n- Performance overhead is critical (consider sampling)\n- Team lacks observability infrastructure (Jaeger, Zipkin)\n\n### When to Use Prometheus\n\n**Use Prometheus When:**\n- Need time-series metrics for monitoring and alerting\n- Building operational dashboards (Grafana)\n- Measuring SLIs for SLO compliance\n- Tracking business metrics (requests\u002Fsec, conversion rates)\n- Kubernetes\u002Fcontainerized environments\n\n**Don't Use Prometheus When:**\n- Need high-cardinality metrics (Prometheus has limits)\n- Require long-term metric storage (use Thanos\u002FCortex)\n- Need push-based metrics (Prometheus is pull-based)\n\n### When to Use slog\n\n**Use slog When:**\n- Go 1.21+ projects (standard library, zero dependencies)\n- Need structured logging with JSON output\n- Want high-performance logging with minimal allocations\n- Integrating with log aggregation systems (Loki, ELK)\n\n**Don't Use slog When:**\n- Go \u003C 1.21 (use zap or zerolog instead)\n- Need complex log routing or filtering (use zap)\n- Require very specific features (audit trails, etc.)\n\n### Sampling Strategy Decision\n\n**Always Sample When:**\n- Development\u002Fstaging environments\n- Total traffic \u003C 100 requests\u002Fsec\n- Debugging specific issues\n\n**Probabilistic Sampling When:**\n- Production with moderate traffic (100-10K req\u002Fsec)\n- Sample rate: 1-10% typically\n\n**Tail-Based Sampling When:**\n- High traffic production (>10K req\u002Fsec)\n- Only sample errors and slow requests\n- Requires tail-sampling processor (OpenTelemetry Collector)\n\n## Anti-Patterns to Avoid\n\n### ❌ Not Propagating Context\n\n**WRONG: Breaking trace context**\n```go\nfunc processOrder(order Order) error {\n    \u002F\u002F Creates new context, loses trace!\n    ctx := context.Background()\n    return validateOrder(ctx, order)\n}\n```\n\n**CORRECT: Propagate context through call chain**\n```go\nfunc processOrder(ctx context.Context, order Order) error {\n    \u002F\u002F Propagates trace context\n    return validateOrder(ctx, order)\n}\n```\n\n### ❌ Cardinality Explosion\n\n**WRONG: Unbounded label values**\n```go\n\u002F\u002F user_id can have millions of values!\nhttpRequests.WithLabelValues(r.Method, r.URL.Path, userID).Inc()\n```\n\n**CORRECT: Use bounded labels**\n```go\n\u002F\u002F Only method and path (bounded values)\nhttpRequests.WithLabelValues(r.Method, r.URL.Path).Inc()\n\u002F\u002F Track user-specific metrics separately if needed\n```\n\n### ❌ Logging Sensitive Data\n\n**WRONG: Exposing PII and secrets**\n```go\nlogger.Info(\"user login\",\n    \"email\", user.Email,        \u002F\u002F PII!\n    \"password\", user.Password,  \u002F\u002F CRITICAL!\n    \"token\", authToken,         \u002F\u002F SECRET!\n)\n```\n\n**CORRECT: Redact sensitive information**\n```go\nlogger.Info(\"user login\",\n    \"user_id\", user.ID,  \u002F\u002F Safe identifier\n    \"method\", \"password\",\n)\n```\n\n### ❌ Not Closing Spans\n\n**WRONG: Span leaks memory**\n```go\nfunc processOrder(ctx context.Context) error {\n    ctx, span := tracer.Start(ctx, \"processOrder\")\n    \u002F\u002F Missing defer span.End()!\n\n    if err := validate(); err != nil {\n        return err  \u002F\u002F Span never closed!\n    }\n\n    return nil\n}\n```\n\n**CORRECT: Always defer span.End()**\n```go\nfunc processOrder(ctx context.Context) error {\n    ctx, span := tracer.Start(ctx, \"processOrder\")\n    defer span.End()  \u002F\u002F Always runs\n\n    if err := validate(); err != nil {\n        span.RecordError(err)\n        return err\n    }\n\n    return nil\n}\n```\n\n### ❌ Synchronous Metric Export\n\n**WRONG: Blocking requests with metric export**\n```go\n\u002F\u002F Synchronous export blocks HTTP handler\nexporter := jaeger.New(jaeger.WithCollectorEndpoint(...))\ntp := sdktrace.NewTracerProvider(\n    sdktrace.WithSyncer(exporter),  \u002F\u002F BAD: Synchronous!\n)\n```\n\n**CORRECT: Use batching for async export**\n```go\n\u002F\u002F Batching exports asynchronously\ntp := sdktrace.NewTracerProvider(\n    sdktrace.WithBatcher(exporter),  \u002F\u002F GOOD: Async batching\n)\n```\n\n### ❌ Missing Graceful Shutdown\n\n**WRONG: Losing traces on shutdown**\n```go\nfunc main() {\n    tp, _ := initTracer(\"service\")\n    \u002F\u002F Missing shutdown - spans lost!\n    http.ListenAndServe(\":8080\", nil)\n}\n```\n\n**CORRECT: Shutdown exporters properly**\n```go\nfunc main() {\n    tp, _ := initTracer(\"service\")\n    defer tp.Shutdown(context.Background())\n\n    \u002F\u002F Handle signals and graceful shutdown\n    server.ListenAndServe()\n}\n```\n\n## Best Practices\n\n1. **Context Propagation**: Always pass `context.Context` through call chains\n2. **Bounded Labels**: Keep metric label cardinality under 1000 combinations\n3. **Sampling**: Use probabilistic sampling in high-traffic production\n4. **Correlation IDs**: Include trace_id in logs for correlation\n5. **Health Checks**: Implement both `\u002Fhealth` (liveness) and `\u002Fready` (readiness)\n6. **Graceful Shutdown**: Flush traces and metrics before exit\n7. **Error Recording**: Use `span.RecordError()` for automatic error tracking\n8. **Metric Naming**: Follow Prometheus naming conventions (`_total`, `_seconds`)\n9. **Log Levels**: Use appropriate levels (Debug, Info, Warn, Error)\n10. **Auto-Instrumentation**: Use middleware for HTTP\u002FgRPC when possible\n\n## Metric Naming Conventions\n\nFollow Prometheus best practices:\n\n**Counter Metrics** (always increasing):\n- `http_requests_total` (not `http_requests`)\n- `payment_transactions_total`\n- `errors_total`\n\n**Gauge Metrics** (can go up or down):\n- `active_connections`\n- `queue_size`\n- `memory_usage_bytes`\n\n**Histogram\u002FSummary Metrics** (observations):\n- `http_request_duration_seconds` (not `_milliseconds`)\n- `db_query_duration_seconds`\n- `response_size_bytes`\n\n**Label Naming**:\n- Use `method`, not `http_method`\n- Use `status`, not `status_code` or `http_status`\n- Use snake_case, not camelCase\n\n## Resources\n\n**Official Documentation:**\n- OpenTelemetry Go: https:\u002F\u002Fopentelemetry.io\u002Fdocs\u002Finstrumentation\u002Fgo\u002F\n- Prometheus Client Library: https:\u002F\u002Fgithub.com\u002Fprometheus\u002Fclient_golang\n- Go slog Package: https:\u002F\u002Fpkg.go.dev\u002Flog\u002Fslog\n\n**Recent Guides (2025):**\n- \"Observability in Go: What Real Engineers Are Saying in 2025\" (Quesma Blog)\n- \"Monitoring Go Apps with OpenTelemetry Metrics\" (Better Stack, 2025)\n- Prometheus Best Practices: https:\u002F\u002Fprometheus.io\u002Fdocs\u002Fpractices\u002Fnaming\u002F\n\n**Related Skills:**\n- **golang-web-frameworks**: HTTP server patterns and middleware\n- **golang-testing-strategies**: Testing instrumented code\n- **verification-before-completion**: Validating observability setup\n\n## Quick Reference\n\n### Initialize OpenTelemetry\n```go\ntp, _ := initTracer(\"service-name\")\ndefer tp.Shutdown(context.Background())\n```\n\n### Create Spans\n```go\nctx, span := otel.Tracer(\"name\").Start(ctx, \"operation\")\ndefer span.End()\nspan.SetAttributes(attribute.String(\"key\", \"value\"))\n```\n\n### Define Metrics\n```go\ncounter := promauto.NewCounterVec(opts, []string{\"label\"})\nhistogram := promauto.NewHistogramVec(opts, []string{\"label\"})\n```\n\n### Structured Logging\n```go\nlogger := slog.With(\"trace_id\", traceID)\nlogger.Info(\"message\", \"key\", value)\n```\n\n### Health Checks\n```go\nhttp.HandleFunc(\"\u002Fhealth\", livenessHandler)\nhttp.HandleFunc(\"\u002Fready\", readinessHandler)\n```\n\n---\n\n**Token Estimate**: ~5,000 tokens (entry point + full content)\n**Version**: 1.0.0\n**Last Updated**: 2025-12-03\n","# Go Observability with OpenTelemetry\n\n## Overview\n\nModern Go applications require comprehensive observability through the three pillars: traces, metrics, and logs. OpenTelemetry provides vendor-neutral instrumentation for distributed tracing, Prometheus offers powerful metrics collection, and Go's slog package (1.21+) delivers structured logging with minimal overhead.\n\n**Key Features:**\n- 🔍 **OpenTelemetry**: Distributed tracing with context propagation\n- 📊 **Prometheus**: Metrics collection with \u002Fmetrics endpoint\n- 📝 **Structured Logging**: slog with JSON formatting and correlation IDs\n- 🎯 **Auto-Instrumentation**: HTTP\u002FgRPC middleware patterns\n- 💚 **Health Checks**: Kubernetes-ready readiness\u002Fliveness probes\n- 🔄 **Graceful Shutdown**: Clean exporter shutdown and signal handling\n\n## When to Use This Skill\n\nActivate this skill when:\n- Instrumenting microservices for production observability\n- Setting up distributed tracing across service boundaries\n- Creating operational dashboards with Prometheus\u002FGrafana\n- Debugging production performance issues or bottlenecks\n- Implementing SLOs and monitoring SLIs\n- Adding observability to existing Go applications\n- Correlating logs, traces, and metrics for debugging\n\n## Core Observability Principles\n\n### The Three Pillars\n\n1. **Traces**: Understand request flow across distributed systems\n2. **Metrics**: Measure system behavior and performance over time\n3. **Logs**: Record discrete events for debugging and audit\n\n### Correlation Strategy\n\nAll three pillars must share common identifiers:\n- **Trace ID**: Links all operations in a request\n- **Span ID**: Identifies specific operation within trace\n- **Request ID**: Correlates logs with traces and metrics\n\n## OpenTelemetry Integration\n\n### Installation\n\n```bash\ngo get go.opentelemetry.io\u002Fotel\ngo get go.opentelemetry.io\u002Fotel\u002Fsdk\ngo get go.opentelemetry.io\u002Fotel\u002Fexporters\u002Fjaeger\ngo get go.opentelemetry.io\u002Fcontrib\u002Finstrumentation\u002Fnet\u002Fhttp\u002Fotelhttp\n```\n\n### Basic Setup\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"log\"\n\n    \"go.opentelemetry.io\u002Fotel\"\n    \"go.opentelemetry.io\u002Fotel\u002Fexporters\u002Fjaeger\"\n    \"go.opentelemetry.io\u002Fotel\u002Fsdk\u002Fresource\"\n    sdktrace \"go.opentelemetry.io\u002Fotel\u002Fsdk\u002Ftrace\"\n    semconv \"go.opentelemetry.io\u002Fotel\u002Fsemconv\u002Fv1.21.0\"\n)\n\nfunc initTracer(serviceName string) (*sdktrace.TracerProvider, error) {\n    \u002F\u002F Create Jaeger exporter\n    exporter, err := jaeger.New(jaeger.WithCollectorEndpoint(\n        jaeger.WithEndpoint(\"http:\u002F\u002Flocalhost:14268\u002Fapi\u002Ftraces\"),\n    ))\n    if err != nil {\n        return nil, err\n    }\n\n    \u002F\u002F Create resource with service name\n    res, err := resource.Merge(\n        resource.Default(),\n        resource.NewWithAttributes(\n            semconv.SchemaURL,\n            semconv.ServiceName(serviceName),\n            semconv.ServiceVersion(\"1.0.0\"),\n        ),\n    )\n    if err != nil {\n        return nil, err\n    }\n\n    \u002F\u002F Create tracer provider\n    tp := sdktrace.NewTracerProvider(\n        sdktrace.WithBatcher(exporter),\n        sdktrace.WithResource(res),\n        sdktrace.WithSampler(sdktrace.AlwaysSample()), \u002F\u002F Use probability sampler in production\n    )\n\n    otel.SetTracerProvider(tp)\n    return tp, nil\n}\n\nfunc main() {\n    tp, err := initTracer(\"order-service\")\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer func() {\n        if err := tp.Shutdown(context.Background()); err != nil {\n            log.Printf(\"Error shutting down tracer: %v\", err)\n        }\n    }()\n\n    \u002F\u002F Application code...\n}\n```\n\n### Creating Spans\n\n```go\nimport (\n    \"context\"\n\n    \"go.opentelemetry.io\u002Fotel\"\n    \"go.opentelemetry.io\u002Fotel\u002Fattribute\"\n    \"go.opentelemetry.io\u002Fotel\u002Fcodes\"\n    \"go.opentelemetry.io\u002Fotel\u002Ftrace\"\n)\n\nfunc ProcessOrder(ctx context.Context, order Order) error {\n    tracer := otel.Tracer(\"order-service\")\n    ctx, span := tracer.Start(ctx, \"ProcessOrder\")\n    defer span.End()\n\n    \u002F\u002F Add attributes\n    span.SetAttributes(\n        attribute.String(\"order.id\", order.ID),\n        attribute.Int(\"order.items\", len(order.Items)),\n        attribute.Float64(\"order.total\", order.Total),\n    )\n\n    \u002F\u002F Validate order (creates child span)\n    if err := validateOrder(ctx, order); err != nil {\n        span.RecordError(err)\n        span.SetStatus(codes.Error, \"validation failed\")\n        return err\n    }\n\n    \u002F\u002F Fulfill order\n    if err := fulfillOrder(ctx, order); err != nil {\n        span.RecordError(err)\n        span.SetStatus(codes.Error, \"fulfillment failed\")\n        return err\n    }\n\n    span.SetStatus(codes.Ok, \"order processed successfully\")\n    return nil\n}\n\nfunc validateOrder(ctx context.Context, order Order) error {\n    _, span := otel.Tracer(\"order-service\").Start(ctx, \"validateOrder\")\n    defer span.End()\n\n    \u002F\u002F Validation logic...\n    return nil\n}\n```\n\n### HTTP Middleware Instrumentation\n\n```go\nimport (\n    \"net\u002Fhttp\"\n\n    \"go.opentelemetry.io\u002Fcontrib\u002Finstrumentation\u002Fnet\u002Fhttp\u002Fotelhttp\"\n)\n\nfunc main() {\n    \u002F\u002F Wrap handler with automatic tracing\n    handler := http.HandlerFunc(orderHandler)\n    wrappedHandler := otelhttp.NewHandler(handler, \"order-handler\")\n\n    http.Handle(\"\u002Forders\", wrappedHandler)\n    http.ListenAndServe(\":8080\", nil)\n}\n\n\u002F\u002F Manual instrumentation for more control\nfunc orderHandler(w http.ResponseWriter, r *http.Request) {\n    ctx := r.Context()\n    tracer := otel.Tracer(\"order-service\")\n\n    ctx, span := tracer.Start(ctx, \"orderHandler\")\n    defer span.End()\n\n    \u002F\u002F Extract order ID from request\n    orderID := r.URL.Query().Get(\"id\")\n    span.SetAttributes(attribute.String(\"order.id\", orderID))\n\n    \u002F\u002F Process order with propagated context\n    order, err := fetchOrder(ctx, orderID)\n    if err != nil {\n        span.RecordError(err)\n        http.Error(w, \"Order not found\", http.StatusNotFound)\n        return\n    }\n\n    \u002F\u002F ... handle response\n}\n```\n\n## Prometheus Metrics\n\n### Installation\n\n```bash\ngo get github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\ngo get github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\u002Fpromhttp\n```\n\n### Metric Types and Patterns\n\n```go\npackage metrics\n\nimport (\n    \"github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\"\n    \"github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\u002Fpromauto\"\n)\n\nvar (\n    \u002F\u002F Counter: Monotonically increasing value\n    httpRequestsTotal = promauto.NewCounterVec(\n        prometheus.CounterOpts{\n            Name: \"http_requests_total\",\n            Help: \"Total number of HTTP requests\",\n        },\n        []string{\"method\", \"path\", \"status\"},\n    )\n\n    \u002F\u002F Gauge: Value that can go up or down\n    activeConnections = promauto.NewGauge(\n        prometheus.GaugeOpts{\n            Name: \"active_connections\",\n            Help: \"Number of active connections\",\n        },\n    )\n\n    \u002F\u002F Histogram: Observations bucketed by value\n    httpRequestDuration = promauto.NewHistogramVec(\n        prometheus.HistogramOpts{\n            Name:    \"http_request_duration_seconds\",\n            Help:    \"HTTP request duration in seconds\",\n            Buckets: prometheus.DefBuckets, \u002F\u002F [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]\n        },\n        []string{\"method\", \"path\"},\n    )\n\n    \u002F\u002F Summary: Similar to histogram but calculates quantiles\n    dbQueryDuration = promauto.NewSummaryVec(\n        prometheus.SummaryOpts{\n            Name:       \"db_query_duration_seconds\",\n            Help:       \"Database query duration\",\n            Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001},\n        },\n        []string{\"query_type\"},\n    )\n)\n```\n\n### Metrics Middleware\n\n```go\nimport (\n    \"net\u002Fhttp\"\n    \"strconv\"\n    \"time\"\n\n    \"github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\u002Fpromhttp\"\n)\n\n\u002F\u002F Metrics middleware that instruments all HTTP handlers\nfunc MetricsMiddleware(next http.Handler) http.Handler {\n    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {\n        start := time.Now()\n\n        \u002F\u002F Track active connections\n        activeConnections.Inc()\n        defer activeConnections.Dec()\n\n        \u002F\u002F Wrap response writer to capture status code\n        rw := &responseWriter{ResponseWriter: w, statusCode: http.StatusOK}\n\n        \u002F\u002F Call next handler\n        next.ServeHTTP(rw, r)\n\n        \u002F\u002F Record metrics\n        duration := time.Since(start).Seconds()\n        httpRequestDuration.WithLabelValues(r.Method, r.URL.Path).Observe(duration)\n        httpRequestsTotal.WithLabelValues(r.Method, r.URL.Path, strconv.Itoa(rw.statusCode)).Inc()\n    })\n}\n\ntype responseWriter struct {\n    http.ResponseWriter\n    statusCode int\n}\n\nfunc (rw *responseWriter) WriteHeader(code int) {\n    rw.statusCode = code\n    rw.ResponseWriter.WriteHeader(code)\n}\n\n\u002F\u002F Expose metrics endpoint\nfunc main() {\n    http.Handle(\"\u002Fmetrics\", promhttp.Handler())\n\n    handler := MetricsMiddleware(http.HandlerFunc(orderHandler))\n    http.Handle(\"\u002Forders\", handler)\n\n    http.ListenAndServe(\":8080\", nil)\n}\n```\n\n### Custom Metrics Example\n\n```go\nfunc ProcessPayment(ctx context.Context, payment Payment) error {\n    timer := prometheus.NewTimer(dbQueryDuration.WithLabelValues(\"payment_insert\"))\n    defer timer.ObserveDuration()\n\n    \u002F\u002F Process payment\n    if err := db.InsertPayment(payment); err != nil {\n        httpRequestsTotal.WithLabelValues(\"POST\", \"\u002Fpayments\", \"500\").Inc()\n        return err\n    }\n\n    httpRequestsTotal.WithLabelValues(\"POST\", \"\u002Fpayments\", \"200\").Inc()\n    return nil\n}\n```\n\n## Structured Logging with slog\n\n### Basic Setup (Go 1.21+)\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"log\u002Fslog\"\n    \"os\"\n)\n\nfunc initLogger() *slog.Logger {\n    \u002F\u002F JSON logger for production\n    handler := slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{\n        Level: slog.LevelInfo,\n        AddSource: true, \u002F\u002F Include file:line information\n    })\n\n    logger := slog.New(handler)\n    slog.SetDefault(logger) \u002F\u002F Set as default logger\n    return logger\n}\n\nfunc main() {\n    logger := initLogger()\n\n    logger.Info(\"service starting\",\n        \"service\", \"order-service\",\n        \"version\", \"1.0.0\",\n        \"port\", 8080,\n    )\n}\n```\n\n### Context-Aware Logging\n\n```go\nimport (\n    \"context\"\n    \"log\u002Fslog\"\n\n    \"go.opentelemetry.io\u002Fotel\u002Ftrace\"\n)\n\n\u002F\u002F Add trace context to logger\nfunc LoggerWithTrace(ctx context.Context) *slog.Logger {\n    span := trace.SpanFromContext(ctx)\n    spanCtx := span.SpanContext()\n\n    return slog.With(\n        \"trace_id\", spanCtx.TraceID().String(),\n        \"span_id\", spanCtx.SpanID().String(),\n    )\n}\n\nfunc HandleRequest(ctx context.Context, req Request) error {\n    logger := LoggerWithTrace(ctx)\n\n    logger.Info(\"processing request\",\n        \"request_id\", req.ID,\n        \"method\", req.Method,\n        \"path\", req.Path,\n    )\n\n    if err := processRequest(ctx, req); err != nil {\n        logger.Error(\"request failed\",\n            \"error\", err,\n            \"duration_ms\", time.Since(req.StartTime).Milliseconds(),\n        )\n        return err\n    }\n\n    logger.Info(\"request completed successfully\",\n        \"duration_ms\", time.Since(req.StartTime).Milliseconds(),\n    )\n    return nil\n}\n```\n\n### Log Levels and Structured Fields\n\n```go\nfunc ProcessOrder(ctx context.Context, order Order) error {\n    logger := LoggerWithTrace(ctx).With(\n        \"order_id\", order.ID,\n        \"user_id\", order.UserID,\n    )\n\n    logger.Debug(\"validating order\", \"items\", len(order.Items))\n\n    if len(order.Items) == 0 {\n        logger.Warn(\"empty order received\")\n        return ErrEmptyOrder\n    }\n\n    logger.Info(\"order validation passed\")\n\n    if err := fulfillOrder(ctx, order); err != nil {\n        logger.Error(\"fulfillment failed\",\n            \"error\", err,\n            slog.Group(\"order_details\",\n                \"total\", order.Total,\n                \"items\", len(order.Items),\n            ),\n        )\n        return err\n    }\n\n    logger.Info(\"order processed successfully\",\n        \"total\", order.Total,\n    )\n    return nil\n}\n```\n\n## Health Checks and Graceful Shutdown\n\n### Health Check Endpoints\n\n```go\nimport (\n    \"context\"\n    \"database\u002Fsql\"\n    \"encoding\u002Fjson\"\n    \"net\u002Fhttp\"\n    \"time\"\n)\n\ntype HealthChecker struct {\n    db *sql.DB\n    \u002F\u002F Add other dependencies\n}\n\ntype HealthStatus struct {\n    Status      string            `json:\"status\"`\n    Version     string            `json:\"version\"`\n    Checks      map[string]string `json:\"checks\"`\n    Timestamp   time.Time         `json:\"timestamp\"`\n}\n\n\u002F\u002F Liveness probe - is the app running?\nfunc (hc *HealthChecker) LivenessHandler(w http.ResponseWriter, r *http.Request) {\n    w.Header().Set(\"Content-Type\", \"application\u002Fjson\")\n    w.WriteHeader(http.StatusOK)\n    json.NewEncoder(w).Encode(map[string]string{\n        \"status\": \"alive\",\n    })\n}\n\n\u002F\u002F Readiness probe - is the app ready to serve traffic?\nfunc (hc *HealthChecker) ReadinessHandler(w http.ResponseWriter, r *http.Request) {\n    ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)\n    defer cancel()\n\n    status := HealthStatus{\n        Status:    \"ready\",\n        Version:   \"1.0.0\",\n        Checks:    make(map[string]string),\n        Timestamp: time.Now(),\n    }\n\n    \u002F\u002F Check database\n    if err := hc.db.PingContext(ctx); err != nil {\n        status.Status = \"not_ready\"\n        status.Checks[\"database\"] = \"unhealthy: \" + err.Error()\n        w.WriteHeader(http.StatusServiceUnavailable)\n    } else {\n        status.Checks[\"database\"] = \"healthy\"\n    }\n\n    \u002F\u002F Add more dependency checks (Redis, external APIs, etc.)\n\n    w.Header().Set(\"Content-Type\", \"application\u002Fjson\")\n    if status.Status == \"ready\" {\n        w.WriteHeader(http.StatusOK)\n    }\n    json.NewEncoder(w).Encode(status)\n}\n```\n\n### Graceful Shutdown\n\n```go\nimport (\n    \"context\"\n    \"net\u002Fhttp\"\n    \"os\"\n    \"os\u002Fsignal\"\n    \"syscall\"\n    \"time\"\n)\n\nfunc main() {\n    \u002F\u002F Initialize tracer\n    tp, err := initTracer(\"order-service\")\n    if err != nil {\n        log.Fatal(err)\n    }\n\n    \u002F\u002F Setup HTTP server\n    server := &http.Server{\n        Addr:    \":8080\",\n        Handler: setupRoutes(),\n    }\n\n    \u002F\u002F Channel for shutdown signals\n    shutdown := make(chan os.Signal, 1)\n    signal.Notify(shutdown, os.Interrupt, syscall.SIGTERM)\n\n    \u002F\u002F Start server in goroutine\n    go func() {\n        slog.Info(\"server starting\", \"port\", 8080)\n        if err := server.ListenAndServe(); err != http.ErrServerClosed {\n            log.Fatal(err)\n        }\n    }()\n\n    \u002F\u002F Wait for shutdown signal\n    \u003C-shutdown\n    slog.Info(\"shutdown signal received\")\n\n    \u002F\u002F Create shutdown context with timeout\n    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)\n    defer cancel()\n\n    \u002F\u002F Shutdown HTTP server\n    slog.Info(\"shutting down HTTP server\")\n    if err := server.Shutdown(ctx); err != nil {\n        slog.Error(\"HTTP server shutdown error\", \"error\", err)\n    }\n\n    \u002F\u002F Shutdown tracer provider (flush spans)\n    slog.Info(\"shutting down tracer\")\n    if err := tp.Shutdown(ctx); err != nil {\n        slog.Error(\"tracer shutdown error\", \"error\", err)\n    }\n\n    slog.Info(\"shutdown complete\")\n}\n```\n\n## Complete Instrumentation Example\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"database\u002Fsql\"\n    \"log\u002Fslog\"\n    \"net\u002Fhttp\"\n    \"os\"\n    \"time\"\n\n    \"github.com\u002Fprometheus\u002Fclient_golang\u002Fprometheus\u002Fpromhttp\"\n    \"go.opentelemetry.io\u002Fcontrib\u002Finstrumentation\u002Fnet\u002Fhttp\u002Fotelhttp\"\n    \"go.opentelemetry.io\u002Fotel\"\n    \"go.opentelemetry.io\u002Fotel\u002Fattribute\"\n)\n\ntype Server struct {\n    db     *sql.DB\n    logger *slog.Logger\n}\n\nfunc (s *Server) orderHandler(w http.ResponseWriter, r *http.Request) {\n    ctx := r.Context()\n\n    \u002F\u002F Get tracer and create span\n    tracer := otel.Tracer(\"order-service\")\n    ctx, span := tracer.Start(ctx, \"orderHandler\")\n    defer span.End()\n\n    \u002F\u002F Create context-aware logger with trace ID\n    logger := s.logger.With(\n        \"trace_id\", span.SpanContext().TraceID().String(),\n        \"request_id\", r.Header.Get(\"X-Request-ID\"),\n    )\n\n    orderID := r.URL.Query().Get(\"id\")\n    span.SetAttributes(attribute.String(\"order.id\", orderID))\n\n    logger.Info(\"fetching order\", \"order_id\", orderID)\n\n    \u002F\u002F Fetch order from database\n    order, err := s.fetchOrder(ctx, orderID)\n    if err != nil {\n        span.RecordError(err)\n        logger.Error(\"failed to fetch order\", \"error\", err)\n        http.Error(w, \"Order not found\", http.StatusNotFound)\n        return\n    }\n\n    logger.Info(\"order fetched successfully\",\n        \"order_id\", orderID,\n        \"items\", len(order.Items),\n    )\n\n    \u002F\u002F Return order as JSON\n    w.Header().Set(\"Content-Type\", \"application\u002Fjson\")\n    json.NewEncoder(w).Encode(order)\n}\n\nfunc (s *Server) fetchOrder(ctx context.Context, orderID string) (*Order, error) {\n    _, span := otel.Tracer(\"order-service\").Start(ctx, \"fetchOrder\")\n    defer span.End()\n\n    \u002F\u002F Time database query\n    start := time.Now()\n\n    var order Order\n    err := s.db.QueryRowContext(ctx, \"SELECT * FROM orders WHERE id = ?\", orderID).Scan(&order)\n\n    duration := time.Since(start).Seconds()\n    dbQueryDuration.WithLabelValues(\"select_order\").Observe(duration)\n\n    return &order, err\n}\n\nfunc setupRoutes(s *Server, hc *HealthChecker) http.Handler {\n    mux := http.NewServeMux()\n\n    \u002F\u002F Health endpoints (no tracing needed)\n    mux.HandleFunc(\"\u002Fhealth\", hc.LivenessHandler)\n    mux.HandleFunc(\"\u002Fready\", hc.ReadinessHandler)\n    mux.Handle(\"\u002Fmetrics\", promhttp.Handler())\n\n    \u002F\u002F Business endpoints (with tracing)\n    orderHandler := http.HandlerFunc(s.orderHandler)\n    mux.Handle(\"\u002Forders\", otelhttp.NewHandler(orderHandler, \"orders\"))\n\n    \u002F\u002F Wrap everything with metrics middleware\n    return MetricsMiddleware(mux)\n}\n```\n\n## Decision Trees\n\n### When to Use OpenTelemetry\n\n**Use OpenTelemetry When:**\n- Building distributed systems with multiple services\n- Need to trace requests across service boundaries\n- Debugging performance issues in microservices\n- Want vendor-neutral observability (switch backends easily)\n- Require correlation between traces, metrics, and logs\n\n**Don't Use OpenTelemetry When:**\n- Building simple monolithic applications\n- Performance overhead is critical (consider sampling)\n- Team lacks observability infrastructure (Jaeger, Zipkin)\n\n### When to Use Prometheus\n\n**Use Prometheus When:**\n- Need time-series metrics for monitoring and alerting\n- Building operational dashboards (Grafana)\n- Measuring SLIs for SLO compliance\n- Tracking business metrics (requests\u002Fsec, conversion rates)\n- Kubernetes\u002Fcontainerized environments\n\n**Don't Use Prometheus When:**\n- Need high-cardinality metrics (Prometheus has limits)\n- Require long-term metric storage (use Thanos\u002FCortex)\n- Need push-based metrics (Prometheus is pull-based)\n\n### When to Use slog\n\n**Use slog When:**\n- Go 1.21+ projects (standard library, zero dependencies)\n- Need structured logging with JSON output\n- Want high-performance logging with minimal allocations\n- Integrating with log aggregation systems (Loki, ELK)\n\n**Don't Use slog When:**\n- Go \u003C 1.21 (use zap or zerolog instead)\n- Need complex log routing or filtering (use zap)\n- Require very specific features (audit trails, etc.)\n\n### Sampling Strategy Decision\n\n**Always Sample When:**\n- Development\u002Fstaging environments\n- Total traffic \u003C 100 requests\u002Fsec\n- Debugging specific issues\n\n**Probabilistic Sampling When:**\n- Production with moderate traffic (100-10K req\u002Fsec)\n- Sample rate: 1-10% typically\n\n**Tail-Based Sampling When:**\n- High traffic production (>10K req\u002Fsec)\n- Only sample errors and slow requests\n- Requires tail-sampling processor (OpenTelemetry Collector)\n\n## Anti-Patterns to Avoid\n\n### ❌ Not Propagating Context\n\n**WRONG: Breaking trace context**\n```go\nfunc processOrder(order Order) error {\n    \u002F\u002F Creates new context, loses trace!\n    ctx := context.Background()\n    return validateOrder(ctx, order)\n}\n```\n\n**CORRECT: Propagate context through call chain**\n```go\nfunc processOrder(ctx context.Context, order Order) error {\n    \u002F\u002F Propagates trace context\n    return validateOrder(ctx, order)\n}\n```\n\n### ❌ Cardinality Explosion\n\n**WRONG: Unbounded label values**\n```go\n\u002F\u002F user_id can have millions of values!\nhttpRequests.WithLabelValues(r.Method, r.URL.Path, userID).Inc()\n```\n\n**CORRECT: Use bounded labels**\n```go\n\u002F\u002F Only method and path (bounded values)\nhttpRequests.WithLabelValues(r.Method, r.URL.Path).Inc()\n\u002F\u002F Track user-specific metrics separately if needed\n```\n\n### ❌ Logging Sensitive Data\n\n**WRONG: Exposing PII and secrets**\n```go\nlogger.Info(\"user login\",\n    \"email\", user.Email,        \u002F\u002F PII!\n    \"password\", user.Password,  \u002F\u002F CRITICAL!\n    \"token\", authToken,         \u002F\u002F SECRET!\n)\n```\n\n**CORRECT: Redact sensitive information**\n```go\nlogger.Info(\"user login\",\n    \"user_id\", user.ID,  \u002F\u002F Safe identifier\n    \"method\", \"password\",\n)\n```\n\n### ❌ Not Closing Spans\n\n**WRONG: Span leaks memory**\n```go\nfunc processOrder(ctx context.Context) error {\n    ctx, span := tracer.Start(ctx, \"processOrder\")\n    \u002F\u002F Missing defer span.End()!\n\n    if err := validate(); err != nil {\n        return err  \u002F\u002F Span never closed!\n    }\n\n    return nil\n}\n```\n\n**CORRECT: Always defer span.End()**\n```go\nfunc processOrder(ctx context.Context) error {\n    ctx, span := tracer.Start(ctx, \"processOrder\")\n    defer span.End()  \u002F\u002F Always runs\n\n    if err := validate(); err != nil {\n        span.RecordError(err)\n        return err\n    }\n\n    return nil\n}\n```\n\n### ❌ Synchronous Metric Export\n\n**WRONG: Blocking requests with metric export**\n```go\n\u002F\u002F Synchronous export blocks HTTP handler\nexporter := jaeger.New(jaeger.WithCollectorEndpoint(...))\ntp := sdktrace.NewTracerProvider(\n    sdktrace.WithSyncer(exporter),  \u002F\u002F BAD: Synchronous!\n)\n```\n\n**CORRECT: Use batching for async export**\n```go\n\u002F\u002F Batching exports asynchronously\ntp := sdktrace.NewTracerProvider(\n    sdktrace.WithBatcher(exporter),  \u002F\u002F GOOD: Async batching\n)\n```\n\n### ❌ Missing Graceful Shutdown\n\n**WRONG: Losing traces on shutdown**\n```go\nfunc main() {\n    tp, _ := initTracer(\"service\")\n    \u002F\u002F Missing shutdown - spans lost!\n    http.ListenAndServe(\":8080\", nil)\n}\n```\n\n**CORRECT: Shutdown exporters properly**\n```go\nfunc main() {\n    tp, _ := initTracer(\"service\")\n    defer tp.Shutdown(context.Background())\n\n    \u002F\u002F Handle signals and graceful shutdown\n    server.ListenAndServe()\n}\n```\n\n## Best Practices\n\n1. **Context Propagation**: Always pass `context.Context` through call chains\n2. **Bounded Labels**: Keep metric label cardinality under 1000 combinations\n3. **Sampling**: Use probabilistic sampling in high-traffic production\n4. **Correlation IDs**: Include trace_id in logs for correlation\n5. **Health Checks**: Implement both `\u002Fhealth` (liveness) and `\u002Fready` (readiness)\n6. **Graceful Shutdown**: Flush traces and metrics before exit\n7. **Error Recording**: Use `span.RecordError()` for automatic error tracking\n8. **Metric Naming**: Follow Prometheus naming conventions (`_total`, `_seconds`)\n9. **Log Levels**: Use appropriate levels (Debug, Info, Warn, Error)\n10. **Auto-Instrumentation**: Use middleware for HTTP\u002FgRPC when possible\n\n## Metric Naming Conventions\n\nFollow Prometheus best practices:\n\n**Counter Metrics** (always increasing):\n- `http_requests_total` (not `http_requests`)\n- `payment_transactions_total`\n- `errors_total`\n\n**Gauge Metrics** (can go up or down):\n- `active_connections`\n- `queue_size`\n- `memory_usage_bytes`\n\n**Histogram\u002FSummary Metrics** (observations):\n- `http_request_duration_seconds` (not `_milliseconds`)\n- `db_query_duration_seconds`\n- `response_size_bytes`\n\n**Label Naming**:\n- Use `method`, not `http_method`\n- Use `status`, not `status_code` or `http_status`\n- Use snake_case, not camelCase\n\n## Resources\n\n**Official Documentation:**\n- OpenTelemetry Go: https:\u002F\u002Fopentelemetry.io\u002Fdocs\u002Finstrumentation\u002Fgo\u002F\n- Prometheus Client Library: https:\u002F\u002Fgithub.com\u002Fprometheus\u002Fclient_golang\n- Go slog Package: https:\u002F\u002Fpkg.go.dev\u002Flog\u002Fslog\n\n**Recent Guides (2025):**\n- \"Observability in Go: What Real Engineers Are Saying in 2025\" (Quesma Blog)\n- \"Monitoring Go Apps with OpenTelemetry Metrics\" (Better Stack, 2025)\n- Prometheus Best Practices: https:\u002F\u002Fprometheus.io\u002Fdocs\u002Fpractices\u002Fnaming\u002F\n\n**Related Skills:**\n- **golang-web-frameworks**: HTTP server patterns and middleware\n- **golang-testing-strategies**: Testing instrumented code\n- **verification-before-completion**: Validating observability setup\n\n## Quick Reference\n\n### Initialize OpenTelemetry\n```go\ntp, _ := initTracer(\"service-name\")\ndefer tp.Shutdown(context.Background())\n```\n\n### Create Spans\n```go\nctx, span := otel.Tracer(\"name\").Start(ctx, \"operation\")\ndefer span.End()\nspan.SetAttributes(attribute.String(\"key\", \"value\"))\n```\n\n### Define Metrics\n```go\ncounter := promauto.NewCounterVec(opts, []string{\"label\"})\nhistogram := promauto.NewHistogramVec(opts, []string{\"label\"})\n```\n\n### Structured Logging\n```go\nlogger := slog.With(\"trace_id\", traceID)\nlogger.Info(\"message\", \"key\", value)\n```\n\n### Health Checks\n```go\nhttp.HandleFunc(\"\u002Fhealth\", livenessHandler)\nhttp.HandleFunc(\"\u002Fready\", readinessHandler)\n```\n\n---\n\n**Token Estimate**: ~5,000 tokens (entry point + full content)\n**Version**: 1.0.0\n**Last Updated**: 2025-12-03\n",{"name":5,"tags":24,"author":33,"license":15,"version":34,"category":35,"description":7,"context_limit":36,"requires_tools":37,"progressive_disclosure":38},[25,26,27,28,29,30,31,32],"observability","golang","opentelemetry","prometheus","tracing","metrics","logging","slog","Claude MPM Team","1.0.0","toolchain",700,[],{"entry_point":39,"token_estimate":43},{"summary":40,"quick_start":41,"when_to_use":42},"Master Go observability through OpenTelemetry distributed tracing, Prometheus metrics, and slog structured logging for production-ready microservices","1. Initialize OpenTelemetry tracer and exporter 2. Add Prometheus metrics endpoint 3. Configure slog for structured logging 4. Instrument HTTP middleware 5. Implement health checks","Instrumenting microservices for observability, setting up distributed tracing across services, creating operational dashboards, debugging production issues, performance monitoring and optimization",{"full":44,"entry":45},5000,150,"main","590aa4fe05d1ec71a02ee9cc76c9a20508b5bcea","skills\u002Fju\u002Fskills\u002Fgolang-observability-opentelemetry","https:\u002F\u002Fgithub.com\u002Fjulio211916\u002FTlantiCAD-Studio-V0.1-alpha","https:\u002F\u002Fgithub.com\u002Fjulio211916","Estudiante de ing sistemas y psicologia","2026-05-22 04:43:54",{"source":8,"skillName":5,"commands":54,"prompt":64,"zipAvailable":65},[55,58,61],{"tool":56,"command":57},"npx","npx skills add julio211916\u002FTlantiCAD-Studio-V0.1-alpha --skill golang-observability-opentelemetry",{"tool":59,"command":60},"bunx","bunx skills add julio211916\u002FTlantiCAD-Studio-V0.1-alpha --skill golang-observability-opentelemetry",{"tool":62,"command":63},"pnpm","pnpm dlx skills add julio211916\u002FTlantiCAD-Studio-V0.1-alpha --skill golang-observability-opentelemetry","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fjulio211916\u002FTlantiCAD-Studio-V0.1-alpha\u002Fblob\u002Fmain\u002Fskills\u002Fju\u002Fskills\u002Fgolang-observability-opentelemetry\u002FSKILL.md",true,[],[68,75,82,89,96,103],{"skillNo":69,"name":70,"nameZh":6,"description":71,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":72,"categories":73,"keywords":74,"publishedAt":19},"01M735PMP7K29TEC1BC5A3VJ3Z","simpy","Process-based discrete-event simulation framework in Python. Use this skill when building simulations of systems with processes, queues, resources, and time-based events such as manufacturing systems, service operations, network traffic, logistics, or any system where entities interact with shared resources over time.","https:\u002F\u002Fgithub.com\u002Fjulio211916\u002FTlantiCAD-Studio-V0.1-alpha\u002Fblob\u002Fmain\u002Fskills\u002Fju\u002Fskills\u002Fsimpy\u002FSKILL.md",[],[],{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":19},"01M735NX56AZHYMV3BQ8GYZ7J0","shopify-expert","Builds and debugs Shopify themes (.liquid files, theme.json, sections), develops custom Shopify apps (shopify.app.toml, OAuth, webhooks), and implements Storefront API integrations for headless storefronts. Use when building or customizing Shopify themes, creating Hydrogen or custom React storefronts, developing Shopify apps, implementing checkout UI extensions or Shopify Functions, optimizing performance, or integrating third-party services. Invoke for Liquid templating, Storefront API, app development, checkout customization, Shopify Plus features, App Bridge, Polaris, or Shopify CLI workflows.","https:\u002F\u002Fgithub.com\u002Fjulio211916\u002FTlantiCAD-Studio-V0.1-alpha\u002Fblob\u002Fmain\u002Fskills\u002Fju\u002Fskills\u002Fshopify-expert\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":19},"01M735N9WTN69TVA2T5VS9W1CA","shap","Model interpretability and explainability using SHAP (SHapley Additive exPlanations). Use this skill when explaining machine learning model predictions, computing feature importance, generating SHAP plots (waterfall, beeswarm, bar, scatter, force, heatmap), debugging models, analyzing model bias or fairness, comparing models, or implementing explainable AI. Works with tree-based models (XGBoost, LightGBM, Random Forest), deep learning (TensorFlow, PyTorch), linear models, and any black-box model.","https:\u002F\u002Fgithub.com\u002Fjulio211916\u002FTlantiCAD-Studio-V0.1-alpha\u002Fblob\u002Fmain\u002Fskills\u002Fju\u002Fskills\u002Fshap\u002FSKILL.md",[],[],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":93,"categories":94,"keywords":95,"publishedAt":19},"01M735N3VQ8R56TXFF16BMF19H","shadcn-ui","Deep expertise on shadcn\u002Fui component library including installation, customization, theming, and accessibility patterns. Use when building UI with shadcn\u002Fui components, integrating Radix UI primitives, configuring Tailwind CSS v4 themes, or implementing dark mode. Covers Next.js App Router, Pages Router, and Vite setups.","https:\u002F\u002Fgithub.com\u002Fjulio211916\u002FTlantiCAD-Studio-V0.1-alpha\u002Fblob\u002Fmain\u002Fskills\u002Fju\u002Fskills\u002Fshadcn-ui\u002FSKILL.md",[],[],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":102,"publishedAt":19},"01M735KH64KS6Z58H6BJWX89BV","seoul-subway-arrival","Look up Seoul real-time subway arrival information with the official Seoul Open Data API. Use when the user asks when a train arrives, which trains are approaching a station, or how crowded Seoul subway timing looks right now.","https:\u002F\u002Fgithub.com\u002Fjulio211916\u002FTlantiCAD-Studio-V0.1-alpha\u002Fblob\u002Fmain\u002Fskills\u002Fju\u002Fskills\u002Fseoul-subway-arrival\u002FSKILL.md",[],[],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":19},"01M735K7KAFBZNACQ66PMDQMQD","seo-optimization","Comprehensive 7-phase SEO optimization workflow covering technical SEO auditing, keyword research strategy, on-page optimization, content SEO and E-E-A-T, structured data implementation, link building, and AI search optimization. Use when auditing search engine visibility, diagnosing ranking issues, implementing schema markup, conducting keyword research, optimizing content for E-E-A-T signals, or adapting for AI-driven search (AI Overviews, SGE).","https:\u002F\u002Fgithub.com\u002Fjulio211916\u002FTlantiCAD-Studio-V0.1-alpha\u002Fblob\u002Fmain\u002Fskills\u002Fju\u002Fskills\u002Fseo-optimization\u002FSKILL.md",[],[]]