[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6F92Z06RJS7G1QFZE2EV2W9":3,"skill-same:01M6F92Z06RJS7G1QFZE2EV2W9":79},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":45,"linkOnly":46,"markdownContent":47,"markdownBody":48,"frontMatter":49,"refName":35,"commitSha":59,"skillDirectory":60,"forksCount":61,"repositoryHtmlUrl":62,"repositoryHomepageUrl":6,"ownerHtmlUrl":63,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":64,"install":65,"resources":78},"01M6F92Z06RJS7G1QFZE2EV2W9","model-serving-kubernetes",null,"Deploy ML models on Kubernetes with KServe (formerly KFServing) and NVIDIA Triton Inference Server.","boisenoise\u002Fskills-collections","boisenoise","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F193180?v=4","01M6CDBDQK6BDN55ZMA6B6PKBM","01M6CDBDVSJ3JPTBYN26FZ7TZK",12,"MIT","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-model-serving-kubernetes\u002FSKILL.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000011","运维与部署","SCENARIO","使用场景",[23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44],"serving","model serving kubernetes","model","model serving","kubernetes","community","2026","relevant","triton","scripts","critical","platform","main","only","access","devops-security","kubectl","NVIDIA","ci","server","git","security","2026-11-10 10:12:30",false,"---\nname: model-serving-kubernetes\ndescription: Deploy ML models on Kubernetes with KServe (formerly KFServing) and NVIDIA\n  Triton Inference Server.\ncategory: devops\nrisk: critical\nsource: https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\nsource_repo: BagelHole\u002FDevOps-Security-Agent-Skills\nsource_type: community\ndate_added: '2026-09-20'\nlicense: MIT\nlicense_source: https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\u002Fblob\u002Fmain\u002FLICENSE\ncompatibility: Requires the relevant platform CLIs (kubectl, helm, terraform, git,\n  CI runners) and authorized access to the target environment. Docs-only; helper scripts\n  and templates not bundled.\nmetadata:\n  author: devops-skills\n  version: '1.0'\n---\n\n# Model Serving on Kubernetes\n\nProduction ML model serving with KServe and Triton — canary deployments, autoscaling, and GPU-aware scheduling.\n\n## When to Use This Skill\n\nUse this skill when:\n- Serving scikit-learn, PyTorch, TensorFlow, or ONNX models at scale\n- Implementing canary deployments and A\u002FB testing for ML models\n- Autoscaling inference pods based on request rate or GPU metrics\n- Deploying LLMs with Triton or KServe on Kubernetes\n- Managing multiple model versions with traffic splitting\n\n## Prerequisites\n\nThese are version-dependent sketches, not a complete cluster installation. Pin chart\u002Fimage versions and check installed CRD schemas and serving mode (Knative versus standard). Canary fields and KEDA targets depend on that mode; target an actual scalable workload supported by the chosen operator, not an assumed InferenceService scale subresource. Configure ingress authentication\u002FTLS, least-privilege model-store access and resource\u002Fcost bounds. Remove or implement the placeholder transformer; arbitrary models and “-v2” IDs must exist and be licensed.\n\n- Kubernetes 1.28+ with GPU nodes\n- KServe installed (or Triton standalone)\n- `kubectl` and `helm` configured\n- NVIDIA GPU Operator installed on cluster\n\n## KServe Installation\n\n```bash\n# Install KServe with Helm\nhelm repo add kserve https:\u002F\u002Fkserve.github.io\u002Fhelm-charts\nhelm repo update\n\nhelm install kserve kserve\u002Fkserve \\\n  --namespace kserve \\\n  --create-namespace \\\n  --set kserve.controller.gateway.ingressGateway.className=nginx\n\n# Verify\nkubectl get pods -n kserve\nkubectl get crd | grep kserve\n```\n\n## Basic InferenceService (KServe)\n\n```yaml\napiVersion: serving.kserve.io\u002Fv1beta1\nkind: InferenceService\nmetadata:\n  name: sklearn-iris\n  namespace: models\nspec:\n  predictor:\n    sklearn:\n      storageUri: gs:\u002F\u002Fkfserving-examples\u002Fmodels\u002Fsklearn\u002F1.0\u002Fmodel\n      resources:\n        requests:\n          cpu: \"1\"\n          memory: 2Gi\n        limits:\n          cpu: \"2\"\n          memory: 4Gi\n```\n\n```bash\nkubectl apply -f inference-service.yaml\n\n# Get inference service URL\nkubectl get inferenceservice sklearn-iris -n models\n# NAME           URL                                          READY   ...\n# sklearn-iris   http:\u002F\u002Fsklearn-iris.models.example.com       True\n\n# Test prediction\ncurl -X POST http:\u002F\u002Fsklearn-iris.models.example.com\u002Fv1\u002Fmodels\u002Fsklearn-iris:predict \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"instances\": [[6.8, 2.8, 4.8, 1.4]]}'\n```\n\n## GPU-Enabled LLM InferenceService\n\n```yaml\napiVersion: serving.kserve.io\u002Fv1beta1\nkind: InferenceService\nmetadata:\n  name: llama-3-8b\n  namespace: models\n  annotations:\n    serving.kserve.io\u002Fenable-prometheus-scraping: \"true\"\nspec:\n  predictor:\n    containers:\n    - name: vllm-container\n      image: vllm\u002Fvllm-openai:latest\n      args:\n      - \"--port\"\n      - \"8080\"\n      - \"--model\"\n      - \"meta-llama\u002FLlama-3.1-8B-Instruct\"\n      - \"--tensor-parallel-size\"\n      - \"1\"\n      - \"--gpu-memory-utilization\"\n      - \"0.90\"\n      ports:\n      - containerPort: 8080\n        protocol: TCP\n      resources:\n        requests:\n          nvidia.com\u002Fgpu: \"1\"\n          memory: \"20Gi\"\n          cpu: \"4\"\n        limits:\n          nvidia.com\u002Fgpu: \"1\"\n          memory: \"24Gi\"\n          cpu: \"8\"\n      readinessProbe:\n        httpGet:\n          path: \u002Fhealth\n          port: 8080\n        initialDelaySeconds: 60\n        periodSeconds: 10\n      env:\n      - name: HUGGING_FACE_HUB_TOKEN\n        valueFrom:\n          secretKeyRef:\n            name: hf-token\n            key: token\n    nodeSelector:\n      nvidia.com\u002Fgpu.present: \"true\"\n  transformer:\n    containers:\n    - name: kserve-container\n      image: kserve\u002Fkserve-transformer:latest\n```\n\n## Canary Deployment (Traffic Splitting)\n\n```yaml\napiVersion: serving.kserve.io\u002Fv1beta1\nkind: InferenceService\nmetadata:\n  name: llama-3-8b\n  namespace: models\nspec:\n  predictor:\n    canaryTrafficPercent: 20    # 20% to new version, 80% to stable\n    containers:\n    - name: vllm-container\n      image: vllm\u002Fvllm-openai:latest\n      args:\n      - \"--model\"\n      - \"meta-llama\u002FLlama-3.1-8B-Instruct-v2\"  # new model version\n      resources:\n        limits:\n          nvidia.com\u002Fgpu: \"1\"\n```\n\n```bash\n# Gradually increase canary traffic\nkubectl patch inferenceservice llama-3-8b -n models \\\n  --type='json' \\\n  -p='[{\"op\":\"replace\",\"path\":\"\u002Fspec\u002Fpredictor\u002FcanaryTrafficPercent\",\"value\":50}]'\n\n# Promote canary to stable\nkubectl patch inferenceservice llama-3-8b -n models \\\n  --type='json' \\\n  -p='[{\"op\":\"remove\",\"path\":\"\u002Fspec\u002Fpredictor\u002FcanaryTrafficPercent\"}]'\n```\n\n## Autoscaling with KEDA\n\n```yaml\napiVersion: keda.sh\u002Fv1alpha1\nkind: ScaledObject\nmetadata:\n  name: llama-scaler\n  namespace: models\nspec:\n  scaleTargetRef:\n    apiVersion: serving.kserve.io\u002Fv1beta1\n    kind: InferenceService\n    name: llama-3-8b\n  minReplicaCount: 1\n  maxReplicaCount: 5\n  triggers:\n  - type: prometheus\n    metadata:\n      serverAddress: http:\u002F\u002Fprometheus-server.monitoring:9090\n      metricName: kserve_request_count\n      threshold: \"10\"\n      query: |\n        sum(rate(kserve_request_count_total{namespace=\"models\",\n            service=\"llama-3-8b\"}[1m]))\n```\n\n## NVIDIA Triton Inference Server\n\n```yaml\napiVersion: apps\u002Fv1\nkind: Deployment\nmetadata:\n  name: triton-server\n  namespace: models\nspec:\n  replicas: 2\n  selector:\n    matchLabels:\n      app: triton\n  template:\n    metadata:\n      labels:\n        app: triton\n    spec:\n      containers:\n      - name: triton\n        image: nvcr.io\u002Fnvidia\u002Ftritonserver:24.05-py3\n        args:\n        - \"tritonserver\"\n        - \"--model-store=s3:\u002F\u002Fmy-model-store\u002Fmodels\"\n        - \"--model-control-mode=poll\"        # auto-load new model versions\n        - \"--repository-poll-secs=30\"\n        - \"--metrics-port=8002\"\n        ports:\n        - containerPort: 8000   # HTTP\n        - containerPort: 8001   # gRPC\n        - containerPort: 8002   # Metrics\n        resources:\n          limits:\n            nvidia.com\u002Fgpu: \"1\"\n        readinessProbe:\n          httpGet:\n            path: \u002Fv2\u002Fhealth\u002Fready\n            port: 8000\n          initialDelaySeconds: 30\n```\n\n## Triton Model Repository Structure\n\n```\ns3:\u002F\u002Fmy-model-store\u002Fmodels\u002F\n├── text-classifier\u002F\n│   ├── config.pbtxt\n│   ├── 1\u002F\n│   │   └── model.onnx\n│   └── 2\u002F\n│       └── model.onnx          # new version; auto-loaded\n├── embedding-model\u002F\n│   ├── config.pbtxt\n│   └── 1\u002F\n│       └── model.onnx\n```\n\n```protobuf\n# config.pbtxt for ONNX model\nname: \"text-classifier\"\nbackend: \"onnxruntime\"\nmax_batch_size: 64\ndynamic_batching {\n  preferred_batch_size: [16, 32]\n  max_queue_delay_microseconds: 1000\n}\ninput [\n  { name: \"input_ids\" data_type: TYPE_INT64 dims: [-1] }\n  { name: \"attention_mask\" data_type: TYPE_INT64 dims: [-1] }\n]\noutput [\n  { name: \"logits\" data_type: TYPE_FP32 dims: [-1] }\n]\ninstance_group [\n  { kind: KIND_GPU count: 2 }   # 2 model instances on GPU\n]\n```\n\n## Model Management Commands\n\n```bash\n# List loaded models (Triton)\ncurl -X POST http:\u002F\u002Ftriton:8000\u002Fv2\u002Frepository\u002Findex\n\n# Load a model (requires explicit model-control mode, unlike poll mode above)\ncurl -X POST http:\u002F\u002Ftriton:8000\u002Fv2\u002Frepository\u002Fmodels\u002Ftext-classifier\u002Fload\n\n# Unload a model (requires explicit model-control mode)\ncurl -X POST http:\u002F\u002Ftriton:8000\u002Fv2\u002Frepository\u002Fmodels\u002Ftext-classifier\u002Funload\n\n# KServe — watch rollout status\nkubectl rollout status deployment\u002Fllama-3-8b-predictor -n models\nkubectl get inferenceservice llama-3-8b -n models -w\n```\n\n## Common Issues\n\n| Issue | Cause | Fix |\n|-------|-------|-----|\n| `InferenceService not ready` | Model loading or OOM | Check predictor pod logs; increase memory limits |\n| Canary stuck at 0% | KNative routing issue | Check `kubectl get ksvc -n models` |\n| Triton missing model | S3 permissions or path | Verify IAM role; check `--model-store` path |\n| Low GPU utilization | Dynamic batching off | Enable `dynamic_batching` in Triton config |\n| Autoscaler not triggering | Prometheus query wrong | Test query in Prometheus UI |\n\n## Best Practices\n\n- Use canary deployments for all model updates — roll back in seconds if metrics degrade.\n- Enable Triton dynamic batching — benchmark throughput and latency for your model and request distribution.\n- Store models in S3\u002FGCS with versioned paths (`s3:\u002F\u002Fbucket\u002Fmodel\u002Fv1\u002F`, `v2\u002F`).\n- Pin GPU node selectors to prevent model pods landing on CPU-only nodes.\n- Monitor p99 latency and error rates per model version during canary rollouts.\n\n## Related Skills\n\n- vllm-server (`vllm-server`) - vLLM for LLM serving\n- llm-inference-scaling (`llm-inference-scaling`) - KEDA autoscaling\n- kubernetes-ops (`kubernetes-ops`) - Core Kubernetes operations\n- gpu-server-management (`gpu-server-management`) - GPU nodes\n\n## Limitations\n\n- Guidance executes against real environments: confirm target, blast radius, and rollback plan before applying anything.\n- Never deploy to production without explicit approval. Docs-only import: upstream scripts and templates not bundled.\n\n### Example\n\n```bash\ngit status && git diff --stat\nkubectl diff -f manifest.yaml\n```\n\n> Adapted from [BagelHole\u002FDevOps-Security-Agent-Skills](https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills) (MIT); frontmatter, When to Use\u002FLimitations, and safety boundaries added for upstream compliance. Docs-only import: helper scripts and templates not bundled.\n","# Model Serving on Kubernetes\n\nProduction ML model serving with KServe and Triton — canary deployments, autoscaling, and GPU-aware scheduling.\n\n## When to Use This Skill\n\nUse this skill when:\n- Serving scikit-learn, PyTorch, TensorFlow, or ONNX models at scale\n- Implementing canary deployments and A\u002FB testing for ML models\n- Autoscaling inference pods based on request rate or GPU metrics\n- Deploying LLMs with Triton or KServe on Kubernetes\n- Managing multiple model versions with traffic splitting\n\n## Prerequisites\n\nThese are version-dependent sketches, not a complete cluster installation. Pin chart\u002Fimage versions and check installed CRD schemas and serving mode (Knative versus standard). Canary fields and KEDA targets depend on that mode; target an actual scalable workload supported by the chosen operator, not an assumed InferenceService scale subresource. Configure ingress authentication\u002FTLS, least-privilege model-store access and resource\u002Fcost bounds. Remove or implement the placeholder transformer; arbitrary models and “-v2” IDs must exist and be licensed.\n\n- Kubernetes 1.28+ with GPU nodes\n- KServe installed (or Triton standalone)\n- `kubectl` and `helm` configured\n- NVIDIA GPU Operator installed on cluster\n\n## KServe Installation\n\n```bash\n# Install KServe with Helm\nhelm repo add kserve https:\u002F\u002Fkserve.github.io\u002Fhelm-charts\nhelm repo update\n\nhelm install kserve kserve\u002Fkserve \\\n  --namespace kserve \\\n  --create-namespace \\\n  --set kserve.controller.gateway.ingressGateway.className=nginx\n\n# Verify\nkubectl get pods -n kserve\nkubectl get crd | grep kserve\n```\n\n## Basic InferenceService (KServe)\n\n```yaml\napiVersion: serving.kserve.io\u002Fv1beta1\nkind: InferenceService\nmetadata:\n  name: sklearn-iris\n  namespace: models\nspec:\n  predictor:\n    sklearn:\n      storageUri: gs:\u002F\u002Fkfserving-examples\u002Fmodels\u002Fsklearn\u002F1.0\u002Fmodel\n      resources:\n        requests:\n          cpu: \"1\"\n          memory: 2Gi\n        limits:\n          cpu: \"2\"\n          memory: 4Gi\n```\n\n```bash\nkubectl apply -f inference-service.yaml\n\n# Get inference service URL\nkubectl get inferenceservice sklearn-iris -n models\n# NAME           URL                                          READY   ...\n# sklearn-iris   http:\u002F\u002Fsklearn-iris.models.example.com       True\n\n# Test prediction\ncurl -X POST http:\u002F\u002Fsklearn-iris.models.example.com\u002Fv1\u002Fmodels\u002Fsklearn-iris:predict \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"instances\": [[6.8, 2.8, 4.8, 1.4]]}'\n```\n\n## GPU-Enabled LLM InferenceService\n\n```yaml\napiVersion: serving.kserve.io\u002Fv1beta1\nkind: InferenceService\nmetadata:\n  name: llama-3-8b\n  namespace: models\n  annotations:\n    serving.kserve.io\u002Fenable-prometheus-scraping: \"true\"\nspec:\n  predictor:\n    containers:\n    - name: vllm-container\n      image: vllm\u002Fvllm-openai:latest\n      args:\n      - \"--port\"\n      - \"8080\"\n      - \"--model\"\n      - \"meta-llama\u002FLlama-3.1-8B-Instruct\"\n      - \"--tensor-parallel-size\"\n      - \"1\"\n      - \"--gpu-memory-utilization\"\n      - \"0.90\"\n      ports:\n      - containerPort: 8080\n        protocol: TCP\n      resources:\n        requests:\n          nvidia.com\u002Fgpu: \"1\"\n          memory: \"20Gi\"\n          cpu: \"4\"\n        limits:\n          nvidia.com\u002Fgpu: \"1\"\n          memory: \"24Gi\"\n          cpu: \"8\"\n      readinessProbe:\n        httpGet:\n          path: \u002Fhealth\n          port: 8080\n        initialDelaySeconds: 60\n        periodSeconds: 10\n      env:\n      - name: HUGGING_FACE_HUB_TOKEN\n        valueFrom:\n          secretKeyRef:\n            name: hf-token\n            key: token\n    nodeSelector:\n      nvidia.com\u002Fgpu.present: \"true\"\n  transformer:\n    containers:\n    - name: kserve-container\n      image: kserve\u002Fkserve-transformer:latest\n```\n\n## Canary Deployment (Traffic Splitting)\n\n```yaml\napiVersion: serving.kserve.io\u002Fv1beta1\nkind: InferenceService\nmetadata:\n  name: llama-3-8b\n  namespace: models\nspec:\n  predictor:\n    canaryTrafficPercent: 20    # 20% to new version, 80% to stable\n    containers:\n    - name: vllm-container\n      image: vllm\u002Fvllm-openai:latest\n      args:\n      - \"--model\"\n      - \"meta-llama\u002FLlama-3.1-8B-Instruct-v2\"  # new model version\n      resources:\n        limits:\n          nvidia.com\u002Fgpu: \"1\"\n```\n\n```bash\n# Gradually increase canary traffic\nkubectl patch inferenceservice llama-3-8b -n models \\\n  --type='json' \\\n  -p='[{\"op\":\"replace\",\"path\":\"\u002Fspec\u002Fpredictor\u002FcanaryTrafficPercent\",\"value\":50}]'\n\n# Promote canary to stable\nkubectl patch inferenceservice llama-3-8b -n models \\\n  --type='json' \\\n  -p='[{\"op\":\"remove\",\"path\":\"\u002Fspec\u002Fpredictor\u002FcanaryTrafficPercent\"}]'\n```\n\n## Autoscaling with KEDA\n\n```yaml\napiVersion: keda.sh\u002Fv1alpha1\nkind: ScaledObject\nmetadata:\n  name: llama-scaler\n  namespace: models\nspec:\n  scaleTargetRef:\n    apiVersion: serving.kserve.io\u002Fv1beta1\n    kind: InferenceService\n    name: llama-3-8b\n  minReplicaCount: 1\n  maxReplicaCount: 5\n  triggers:\n  - type: prometheus\n    metadata:\n      serverAddress: http:\u002F\u002Fprometheus-server.monitoring:9090\n      metricName: kserve_request_count\n      threshold: \"10\"\n      query: |\n        sum(rate(kserve_request_count_total{namespace=\"models\",\n            service=\"llama-3-8b\"}[1m]))\n```\n\n## NVIDIA Triton Inference Server\n\n```yaml\napiVersion: apps\u002Fv1\nkind: Deployment\nmetadata:\n  name: triton-server\n  namespace: models\nspec:\n  replicas: 2\n  selector:\n    matchLabels:\n      app: triton\n  template:\n    metadata:\n      labels:\n        app: triton\n    spec:\n      containers:\n      - name: triton\n        image: nvcr.io\u002Fnvidia\u002Ftritonserver:24.05-py3\n        args:\n        - \"tritonserver\"\n        - \"--model-store=s3:\u002F\u002Fmy-model-store\u002Fmodels\"\n        - \"--model-control-mode=poll\"        # auto-load new model versions\n        - \"--repository-poll-secs=30\"\n        - \"--metrics-port=8002\"\n        ports:\n        - containerPort: 8000   # HTTP\n        - containerPort: 8001   # gRPC\n        - containerPort: 8002   # Metrics\n        resources:\n          limits:\n            nvidia.com\u002Fgpu: \"1\"\n        readinessProbe:\n          httpGet:\n            path: \u002Fv2\u002Fhealth\u002Fready\n            port: 8000\n          initialDelaySeconds: 30\n```\n\n## Triton Model Repository Structure\n\n```\ns3:\u002F\u002Fmy-model-store\u002Fmodels\u002F\n├── text-classifier\u002F\n│   ├── config.pbtxt\n│   ├── 1\u002F\n│   │   └── model.onnx\n│   └── 2\u002F\n│       └── model.onnx          # new version; auto-loaded\n├── embedding-model\u002F\n│   ├── config.pbtxt\n│   └── 1\u002F\n│       └── model.onnx\n```\n\n```protobuf\n# config.pbtxt for ONNX model\nname: \"text-classifier\"\nbackend: \"onnxruntime\"\nmax_batch_size: 64\ndynamic_batching {\n  preferred_batch_size: [16, 32]\n  max_queue_delay_microseconds: 1000\n}\ninput [\n  { name: \"input_ids\" data_type: TYPE_INT64 dims: [-1] }\n  { name: \"attention_mask\" data_type: TYPE_INT64 dims: [-1] }\n]\noutput [\n  { name: \"logits\" data_type: TYPE_FP32 dims: [-1] }\n]\ninstance_group [\n  { kind: KIND_GPU count: 2 }   # 2 model instances on GPU\n]\n```\n\n## Model Management Commands\n\n```bash\n# List loaded models (Triton)\ncurl -X POST http:\u002F\u002Ftriton:8000\u002Fv2\u002Frepository\u002Findex\n\n# Load a model (requires explicit model-control mode, unlike poll mode above)\ncurl -X POST http:\u002F\u002Ftriton:8000\u002Fv2\u002Frepository\u002Fmodels\u002Ftext-classifier\u002Fload\n\n# Unload a model (requires explicit model-control mode)\ncurl -X POST http:\u002F\u002Ftriton:8000\u002Fv2\u002Frepository\u002Fmodels\u002Ftext-classifier\u002Funload\n\n# KServe — watch rollout status\nkubectl rollout status deployment\u002Fllama-3-8b-predictor -n models\nkubectl get inferenceservice llama-3-8b -n models -w\n```\n\n## Common Issues\n\n| Issue | Cause | Fix |\n|-------|-------|-----|\n| `InferenceService not ready` | Model loading or OOM | Check predictor pod logs; increase memory limits |\n| Canary stuck at 0% | KNative routing issue | Check `kubectl get ksvc -n models` |\n| Triton missing model | S3 permissions or path | Verify IAM role; check `--model-store` path |\n| Low GPU utilization | Dynamic batching off | Enable `dynamic_batching` in Triton config |\n| Autoscaler not triggering | Prometheus query wrong | Test query in Prometheus UI |\n\n## Best Practices\n\n- Use canary deployments for all model updates — roll back in seconds if metrics degrade.\n- Enable Triton dynamic batching — benchmark throughput and latency for your model and request distribution.\n- Store models in S3\u002FGCS with versioned paths (`s3:\u002F\u002Fbucket\u002Fmodel\u002Fv1\u002F`, `v2\u002F`).\n- Pin GPU node selectors to prevent model pods landing on CPU-only nodes.\n- Monitor p99 latency and error rates per model version during canary rollouts.\n\n## Related Skills\n\n- vllm-server (`vllm-server`) - vLLM for LLM serving\n- llm-inference-scaling (`llm-inference-scaling`) - KEDA autoscaling\n- kubernetes-ops (`kubernetes-ops`) - Core Kubernetes operations\n- gpu-server-management (`gpu-server-management`) - GPU nodes\n\n## Limitations\n\n- Guidance executes against real environments: confirm target, blast radius, and rollback plan before applying anything.\n- Never deploy to production without explicit approval. Docs-only import: upstream scripts and templates not bundled.\n\n### Example\n\n```bash\ngit status && git diff --stat\nkubectl diff -f manifest.yaml\n```\n\n> Adapted from [BagelHole\u002FDevOps-Security-Agent-Skills](https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills) (MIT); frontmatter, When to Use\u002FLimitations, and safety boundaries added for upstream compliance. Docs-only import: helper scripts and templates not bundled.\n",{"name":5,"risk":33,"source":50,"license":14,"category":51,"metadata":52,"date_added":55,"description":7,"source_repo":56,"source_type":28,"compatibility":57,"license_source":58},"https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills","devops",{"author":53,"version":54},"devops-skills","1.0","2026-09-20","BagelHole\u002FDevOps-Security-Agent-Skills","Requires the relevant platform CLIs (kubectl, helm, terraform, git, CI runners) and authorized access to the target environment. Docs-only; helper scripts and templates not bundled.","https:\u002F\u002Fgithub.com\u002FBagelHole\u002FDevOps-Security-Agent-Skills\u002Fblob\u002Fmain\u002FLICENSE","7bf2279c8a65758319526a721d5c4b75683aec9c","skills\u002Fantigravity-model-serving-kubernetes",3,"https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections","https:\u002F\u002Fgithub.com\u002Fboisenoise","2026-11-11 14:20:43",{"source":8,"skillName":5,"commands":66,"prompt":76,"zipAvailable":77},[67,70,73],{"tool":68,"command":69},"npx","npx skills add boisenoise\u002Fskills-collections --skill model-serving-kubernetes",{"tool":71,"command":72},"bunx","bunx skills add boisenoise\u002Fskills-collections --skill model-serving-kubernetes",{"tool":74,"command":75},"pnpm","pnpm dlx skills add boisenoise\u002Fskills-collections --skill model-serving-kubernetes","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-model-serving-kubernetes\u002FSKILL.md",true,[],[80,105,131,157,176,199],{"skillNo":81,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":84,"licenseFileUrl":85,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":45},"01M6WNH92BHVFXHGPDZCPJ0AG0","web-access","Use Treg for public web or platform data only when existing search, browser and connected tools cannot cover the task, with explicit cost and data boundaries.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-web-access\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-web-access\u002FSKILL.md",[],[89,37,90,91,92,93,94,95,96,97,98,36,99,100,34,33,101,102,103,104,29],"web","web access","official","development","browser","search","data","Tools","read","fetch","youtube","free tools","free","cover","normal","when",{"skillNo":106,"name":107,"nameZh":6,"description":108,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":84,"licenseFileUrl":109,"sourceHtmlUrl":110,"categories":111,"keywords":115,"publishedAt":45},"01M6WNF9XAYAB2CZDDJBAKNM93","treg","Use the Treg CLI or MCP catalog to discover, price and call authorized external tools with team-scoped credentials and a user-approved budget.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-treg\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-treg\u002FSKILL.md",[112],{"categoryNo":113,"categoryName":114,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000023","Skill与智能体开发",[107,116,117,118,91,92,119,96,120,121,122,123,124,125,126,33,127,128,129,130,29],"MCP","cli","github","license","Budget","credentials","team","catalog","use","call","discover","external tools","with","scoped","blob",{"skillNo":132,"name":133,"nameZh":6,"description":134,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":84,"licenseFileUrl":135,"sourceHtmlUrl":136,"categories":137,"keywords":143,"publishedAt":45},"01M6WN93MEAZ4E41DYAHJRPHQT","recruit","Turn a job description into an evidence-linked candidate shortlist through Treg, with job-related ranking and optional authorized business contact enrichment.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-recruit\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-recruit\u002FSKILL.md",[138],{"categoryNo":139,"categoryName":140,"taxonomyType":141,"taxonomyTypeDesc":142},"01JBSEEDCAT000000000000038","人力资源","PROFESSION","适用职业",[133,118,91,119,144,145,146,147,148,149,33,150,151,152,128,153,154,155,156,130,107,29],"https","email","enrichment","business","evidence","phone","ranking","post","job description","for","person","candidate","reason",{"skillNo":158,"name":159,"nameZh":6,"description":160,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":84,"licenseFileUrl":161,"sourceHtmlUrl":162,"categories":163,"keywords":164,"publishedAt":45},"01M6WN8R5RXSMHM9B4D9FAFX42","rea-growth","Analyze a company’s public growth and content through Treg, distinguish observations from hypotheses, and produce an evidence-linked playbook within an approved budget.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-rea-growth\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-rea-growth\u002FSKILL.md",[],[165,166,167,118,91,119,168,169,144,170,120,171,172,173,148,33,174,175,130,107,29],"growth","rea","rea growth","produce","content","playbook","marketing","analyze","public","observations","company",{"skillNo":177,"name":178,"nameZh":6,"description":179,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":84,"licenseFileUrl":180,"sourceHtmlUrl":181,"categories":182,"keywords":183,"publishedAt":45},"01M6WN2EVE3RQRGWAKBT4S7YT2","make-ugc","Create original AI UGC-style product videos through Treg: research hooks, approve a licensed presenter and voice, generate clips, add captions and report costs.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-make-ugc\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-make-ugc\u002FSKILL.md",[],[184,185,118,186,91,187,119,188,189,190,191,192,33,193,194,195,196,197,198,130],"make","make ugc","research","create","Video","Captions","ads","short","hooks","head","style","niche","trending","talking-head","approve",{"skillNo":200,"name":201,"nameZh":6,"description":202,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":84,"licenseFileUrl":203,"sourceHtmlUrl":204,"categories":205,"keywords":206,"publishedAt":45},"01M6WN10ARAA63Y8K1YNGXJ2RM","lead-signals","Find and qualify public buying signals through Treg, rank fitting business accounts and optionally enrich authorized business contact information.","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-lead-signals\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fboisenoise\u002Fskills-collections\u002Fblob\u002Fmain\u002Fskills\u002Fantigravity-lead-signals\u002FSKILL.md",[],[207,208,209,118,91,119,210,211,147,212,213,33,214,215,216,217,218,219,220,130],"signals","lead","lead signals","how","find","ready","as","how to","qualify","people","information","enrich","changes","funding"]