[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6N0E3R2RT3A8F8DWDB9MHFC":3,"skill-same:01M6N0E3R2RT3A8F8DWDB9MHFC":119},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":22,"publishedAt":43,"linkOnly":44,"markdownContent":45,"markdownBody":46,"frontMatter":47,"refName":55,"commitSha":56,"skillDirectory":57,"forksCount":58,"repositoryHtmlUrl":59,"repositoryHomepageUrl":6,"ownerHtmlUrl":60,"ownerBlogUrl":6,"ownerDescription":61,"updatedAt":62,"install":63,"resources":76},"01M6N0E3R2RT3A8F8DWDB9MHFC","swift-mlx-lm",null,"MLX Swift LM - Run LLMs and VLMs on Apple Silicon using MLX. Covers local inference, streaming, tool calling, LoRA fine-tuning, and embeddings.","Kernel8901\u002Fai-agent-skills-classification","Kernel8901","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F72647553?v=4","01M6KV388J2H5PNC8D8P940BJD","01M6KV38CS5B3AY63RKZB410YM",7,"MIT","https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Fskill.md",[17],{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000023","Skill与智能体开发","SCENARIO","使用场景",[23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42],"mlx lm","swift mlx lm","swift","mlx","model","silicon","apple","local","tool calling","local llm","calling","vision","tool","apple silicon","tuning","embeddings","streaming","run","training","llms","2026-11-10 10:12:30",false,"---\nname: swift-mlx-lm\ndescription: MLX Swift LM - Run LLMs and VLMs on Apple Silicon using MLX. Covers local inference, streaming, tool calling, LoRA fine-tuning, and embeddings.\ntriggers:\n  - mlx\n  - mlx-swift\n  - mlx-lm\n  - apple silicon llm\n  - local llm swift\n  - vision language model swift\n  - lora training swift\n---\n\n# mlx-swift-lm Skill\n\n## 1. Overview & Triggers\n\nmlx-swift-lm is a Swift package for running Large Language Models (LLMs) and Vision-Language Models (VLMs) on Apple Silicon using MLX. It supports local inference, fine-tuning via LoRA\u002FDoRA, and embeddings generation.\n\n### When to Use This Skill\n- Running LLM\u002FVLM inference on macOS\u002FiOS with Apple Silicon\n- Streaming text generation from local models\n- Vision tasks with images\u002Fvideo (VLMs)\n- Tool calling \u002F function calling with models\n- LoRA adapter training and fine-tuning\n- Text embeddings for RAG\u002Fsemantic search\n\n### Architecture Overview\n```\nMLXLMCommon     - Core infrastructure (ModelContainer, ChatSession, KVCache, etc.)\nMLXLLM          - Text-only LLM support (Llama, Qwen, Gemma, Phi, DeepSeek, etc. - examples, not exhaustive)\nMLXVLM          - Vision-Language Models (Qwen2-VL, PaliGemma, Gemma3, etc. - examples, not exhaustive)\nEmbedders       - Embedding models (BGE, Nomic, MiniLM)\n```\n\n## 2. Key File Reference\n\n| Purpose | File Path |\n|---------|-----------|\n| Thread-safe model wrapper | `Libraries\u002FMLXLMCommon\u002FModelContainer.swift` |\n| Simplified chat API | `Libraries\u002FMLXLMCommon\u002FChatSession.swift` |\n| Generation & streaming | `Libraries\u002FMLXLMCommon\u002FEvaluate.swift` |\n| KV cache types | `Libraries\u002FMLXLMCommon\u002FKVCache.swift` |\n| Model configuration | `Libraries\u002FMLXLMCommon\u002FModelConfiguration.swift` |\n| Chat message types | `Libraries\u002FMLXLMCommon\u002FChat.swift` |\n| Tool call processing | `Libraries\u002FMLXLMCommon\u002FTool\u002FToolCallFormat.swift` |\n| Concurrency utilities | `Libraries\u002FMLXLMCommon\u002FUtilities\u002FSerialAccessContainer.swift` |\n| LLM factory & registry | `Libraries\u002FMLXLLM\u002FLLMModelFactory.swift` |\n| VLM factory & registry | `Libraries\u002FMLXVLM\u002FVLMModelFactory.swift` |\n| LoRA configuration | `Libraries\u002FMLXLMCommon\u002FAdapters\u002FLoRA\u002FLoRAContainer.swift` |\n| LoRA training | `Libraries\u002FMLXLLM\u002FLoraTrain.swift` |\n\n## 3. Quick Start\n\n### LLM Chat (Simplest API)\n\n```swift\nimport MLXLLM\nimport MLXLMCommon\n\n\u002F\u002F Load model (downloads from HuggingFace automatically)\nlet modelContainer = try await LLMModelFactory.shared.loadContainer(\n    configuration: .init(id: \"mlx-community\u002FQwen3-4B-4bit\")\n)\n\n\u002F\u002F Create chat session\nlet session = ChatSession(modelContainer)\n\n\u002F\u002F Single response\nlet response = try await session.respond(to: \"What is Swift?\")\nprint(response)\n\n\u002F\u002F Streaming response\nfor try await chunk in session.streamResponse(to: \"Explain concurrency\") {\n    print(chunk, terminator: \"\")\n}\n```\n\n### VLM with Image\n\n```swift\nimport MLXVLM\nimport MLXLMCommon\n\nlet modelContainer = try await VLMModelFactory.shared.loadContainer(\n    configuration: .init(id: \"mlx-community\u002FQwen2-VL-2B-Instruct-4bit\")\n)\n\nlet session = ChatSession(modelContainer)\n\n\u002F\u002F With image (video is also an optional parameter)\nlet image = UserInput.Image.url(imageURL)\nlet response = try await session.respond(\n    to: \"Describe this image\",\n    image: image,\n    video: nil  \u002F\u002F Optional video parameter\n)\n```\n\n### Embeddings\n\n```swift\nimport Embedders\n\n\u002F\u002F Note: Embedders uses loadModelContainer() helper (not a factory pattern)\nlet container = try await loadModelContainer(\n    configuration: ModelConfiguration(id: \"mlx-community\u002Fbge-small-en-v1.5-mlx\")\n)\n\nlet embeddings = await container.perform { model, tokenizer, pooler in\n    let tokens = tokenizer.encode(text: \"Hello world\")\n    let input = MLXArray(tokens).expandedDimensions(axis: 0)\n    let output = model(input)\n    let pooled = pooler(output, normalize: true)\n    eval(pooled)\n    return pooled\n}\n```\n\n## 4. Primary Workflow: LLM Inference\n\n### ChatSession API (Recommended)\n\n`ChatSession` manages conversation history and KV cache automatically:\n\n```swift\nlet session = ChatSession(\n    modelContainer,\n    instructions: \"You are a helpful assistant\",  \u002F\u002F System prompt\n    generateParameters: GenerateParameters(\n        maxTokens: 500,\n        temperature: 0.7\n    )\n)\n\n\u002F\u002F Multi-turn conversation (history preserved automatically)\nlet r1 = try await session.respond(to: \"What is 2+2?\")\nlet r2 = try await session.respond(to: \"And if you multiply that by 3?\")\n\n\u002F\u002F Clear session to start fresh\nawait session.clear()\n```\n\n### Streaming with generate()\n\nFor lower-level control, use `generate()` directly:\n\n```swift\nlet input = try await modelContainer.prepare(input: UserInput(prompt: .text(\"Hello\")))\nlet stream = try await modelContainer.generate(input: input, parameters: GenerateParameters())\n\nfor await generation in stream {\n    switch generation {\n    case .chunk(let text):\n        print(text, terminator: \"\")\n    case .info(let info):\n        print(\"\\n\\(info.tokensPerSecond) tok\u002Fs\")\n    case .toolCall(let call):\n        \u002F\u002F Handle tool call\n        break\n    }\n}\n```\n\n### Tool Calling\n\n```swift\n\u002F\u002F 1. Define tool\nstruct WeatherInput: Codable { let location: String }\nstruct WeatherOutput: Codable { let temperature: Double; let conditions: String }\n\nlet weatherTool = Tool\u003CWeatherInput, WeatherOutput>(\n    name: \"get_weather\",\n    description: \"Get current weather\",\n    parameters: [.required(\"location\", type: .string, description: \"City name\")]\n) { input in\n    WeatherOutput(temperature: 22.0, conditions: \"Sunny\")\n}\n\n\u002F\u002F 2. Include tool schema in request\nlet input = UserInput(\n    prompt: .text(\"What's the weather in Tokyo?\"),\n    tools: [weatherTool.schema]\n)\n\n\u002F\u002F 3. Handle tool calls in generation stream\nfor await generation in try await modelContainer.generate(input: input, parameters: params) {\n    switch generation {\n    case .chunk(let text): print(text)\n    case .toolCall(let call):\n        let result = try await call.execute(with: weatherTool)\n        print(\"Weather: \\(result.conditions)\")\n    case .info: break\n    }\n}\n```\n\nSee [references\u002Ftool-calling.md](references\u002Ftool-calling.md) for multi-turn and feeding results back.\n\n### GenerateParameters\n\n```swift\nlet params = GenerateParameters(\n    maxTokens: 1000,           \u002F\u002F nil = unlimited\n    maxKVSize: 4096,           \u002F\u002F Sliding window (uses RotatingKVCache)\n    kvBits: 4,                 \u002F\u002F Quantized cache (4 or 8 bit)\n    temperature: 0.7,          \u002F\u002F 0 = greedy\u002Fargmax\n    topP: 0.9,                 \u002F\u002F Nucleus sampling\n    repetitionPenalty: 1.1,    \u002F\u002F Penalize repeats\n    repetitionContextSize: 20  \u002F\u002F Window for penalty\n)\n```\n\n### Prompt Caching \u002F History Re-hydration\n\nRestore chat from persisted history:\n\n```swift\nlet history: [Chat.Message] = [\n    .system(\"You are helpful\"),\n    .user(\"Hello\"),\n    .assistant(\"Hi there!\")\n]\n\nlet session = ChatSession(\n    modelContainer,\n    history: history\n)\n\u002F\u002F Continues from this point\n```\n\n## 5. Secondary Workflow: VLM Inference\n\n### Image Input Types\n\n```swift\n\u002F\u002F From URL (file or remote)\nlet image = UserInput.Image.url(fileURL)\n\n\u002F\u002F From CIImage\nlet image = UserInput.Image.ciImage(ciImage)\n\n\u002F\u002F From MLXArray directly\nlet image = UserInput.Image.array(mlxArray)\n```\n\n### Video Input\n\n```swift\n\u002F\u002F From URL (file or remote)\nlet video = UserInput.Video.url(videoURL)\n\n\u002F\u002F From AVFoundation asset\nlet video = UserInput.Video.avAsset(avAsset)\n\n\u002F\u002F From pre-extracted frames\nlet video = UserInput.Video.frames(videoFrames)\n\nlet response = try await session.respond(\n    to: \"What happens in this video?\",\n    video: video\n)\n```\n\n### Multiple Images\n\n```swift\nlet images: [UserInput.Image] = [\n    .url(url1),\n    .url(url2)\n]\n\nlet response = try await session.respond(\n    to: \"Compare these two images\",\n    images: images,\n    videos: []\n)\n```\n\n### VLM-Specific Processing\n\n```swift\nlet session = ChatSession(\n    modelContainer,\n    processing: UserInput.Processing(\n        resize: CGSize(width: 512, height: 512)  \u002F\u002F Resize images\n    )\n)\n```\n\n## 6. Best Practices\n\n### DO\n\n```swift\n\u002F\u002F DO: Use ChatSession for multi-turn conversations\nlet session = ChatSession(modelContainer)\n\n\u002F\u002F DO: Use AsyncStream APIs (modern, Swift concurrency)\nfor try await chunk in session.streamResponse(to: prompt) { ... }\n\n\u002F\u002F DO: Check Task.isCancelled in long-running loops\nfor try await generation in stream {\n    if Task.isCancelled { break }\n    \u002F\u002F process generation\n}\n\n\u002F\u002F DO: Use ModelContainer.perform() for thread-safe access\nawait modelContainer.perform { context in\n    \u002F\u002F Access model, tokenizer safely\n    let tokens = try context.tokenizer.applyChatTemplate(messages: messages)\n    return tokens\n}\n\n\u002F\u002F DO: When breaking early from generation, use generateTask() to get a task handle\n\u002F\u002F This is the lower-level API used internally by ChatSession\nlet (stream, task) = generateTask(...)  \u002F\u002F Returns (AsyncStream, Task)\n\nfor await item in stream {\n    if shouldStop { break }\n}\nawait task.value  \u002F\u002F Ensures KV cache cleanup before next generation\n```\n\n> `generateTask()` is defined in `Evaluate.swift`. Most users should use `ChatSession` which handles this internally.\n\n### DON'T\n\n```swift\n\u002F\u002F DON'T: Share MLXArray across tasks (not Sendable)\nlet array = MLXArray(...)\nTask { array.sum() }  \u002F\u002F Wrong!\n\n\u002F\u002F DON'T: Use deprecated callback-based generation\n\u002F\u002F Old:\ngenerate(input: input, parameters: params) { tokens in ... }  \u002F\u002F Deprecated\n\u002F\u002F New:\nfor await generation in try generate(input: input, parameters: params, context: context) { ... }\n\n\u002F\u002F DON'T: Use old perform(model, tokenizer) signature\n\u002F\u002F Old:\nmodelContainer.perform { model, tokenizer in ... }  \u002F\u002F Deprecated\n\u002F\u002F New:\nmodelContainer.perform { context in ... }\n\n\u002F\u002F DON'T: Forget to eval() MLXArrays before returning from perform()\nawait modelContainer.perform { context in\n    let result = context.model(input)\n    eval(result)  \u002F\u002F Required before returning\n    return result.item(Float.self)\n}\n```\n\n### Thread Safety\n\n- `ModelContainer` is `Sendable` and thread-safe\n- `ChatSession` is NOT thread-safe (use from single task)\n- `MLXArray` is NOT `Sendable` - don't pass across isolation boundaries\n- Use `SendableBox` for transferring non-Sendable data in consuming contexts\n\n### Memory Management\n\n```swift\n\u002F\u002F For long contexts, use sliding window cache\nlet params = GenerateParameters(maxKVSize: 4096)\n\n\u002F\u002F For memory efficiency, use quantized cache\nlet params = GenerateParameters(kvBits: 4)  \u002F\u002F or 8\n\n\u002F\u002F Clear session cache when done\nawait session.clear()\n```\n\n## 7. Reference Links\n\nFor detailed documentation on specific topics, see:\n\n| Reference | When to Use |\n|-----------|-------------|\n| [references\u002Fmodel-container.md](references\u002Fmodel-container.md) | Loading models, ModelContainer API, ModelConfiguration |\n| [references\u002Fkv-cache.md](references\u002Fkv-cache.md) | Cache types, memory optimization, cache serialization |\n| [references\u002Fconcurrency.md](references\u002Fconcurrency.md) | Thread safety, SerialAccessContainer, async patterns |\n| [references\u002Ftool-calling.md](references\u002Ftool-calling.md) | Function calling, tool formats, ToolCallProcessor |\n| [references\u002Ftokenizer-chat.md](references\u002Ftokenizer-chat.md) | Tokenizer, Chat.Message, EOS tokens |\n| [references\u002Fsupported-models.md](references\u002Fsupported-models.md) | Model families, registries, model-specific config |\n| [references\u002Flora-adapters.md](references\u002Flora-adapters.md) | LoRA\u002FDoRA\u002FQLoRA, loading adapters |\n| [references\u002Ftraining.md](references\u002Ftraining.md) | LoRATrain API, fine-tuning |\n| [references\u002Fembeddings.md](references\u002Fembeddings.md) | EmbeddingModel, pooling, use cases |\n\n## 8. Deprecated Patterns Summary\n\nMost common migrations (see individual reference files for topic-specific deprecations):\n\n| If you see... | Use instead... |\n|---------------|----------------|\n| `generate(... didGenerate:)` callback | `generate(...) -> AsyncStream` |\n| `perform { model, tokenizer in }` | `perform { context in }` |\n| `TokenIterator(prompt: MLXArray)` | `TokenIterator(input: LMInput)` |\n| `ModelRegistry` typealias | `LLMRegistry` or `VLMRegistry` |\n| `createAttentionMask(h:cache:[KVCache]?)` | `createAttentionMask(h:cache:KVCache?)` |\n\nEach reference file contains a \"Deprecated Patterns\" section with topic-specific migrations.\n\n## 9. Automatic vs Manual Configuration\n\n### Automatic Behaviors (NO developer action needed)\n\nThe framework handles these automatically:\n\n| Feature | Details |\n|---------|---------|\n| **EOS token loading** | Loaded from `config.json` |\n| **EOS token override** | Priority: `generation_config.json` > `config.json` > defaults |\n| **EOS token merging** | All sources merged at generation time |\n| **EOS token detection** | Stops generation automatically when EOS encountered |\n| **Chat template application** | Applied automatically via `applyChatTemplate()` |\n| **Tool call format detection** | Inferred from `model_type` in `config.json` |\n| **Cache type selection** | Based on GenerateParameters (`maxKVSize`, `kvBits`) |\n| **Tokenizer loading** | Loaded from `tokenizer.json` automatically |\n| **Model weights loading** | Downloaded and loaded from HuggingFace |\n\n### Optional Configuration (Developer MAY configure)\n\n| Feature | When to Configure |\n|---------|-------------------|\n| `extraEOSTokens` | Only if model has unlisted stop tokens |\n| `toolCallFormat` | Only to override auto-detection |\n| `maxKVSize` | To enable sliding window cache |\n| `kvBits` | To enable quantized cache (4 or 8 bit) |\n| `maxTokens` | To limit output length |\n","# mlx-swift-lm Skill\n\n## 1. Overview & Triggers\n\nmlx-swift-lm is a Swift package for running Large Language Models (LLMs) and Vision-Language Models (VLMs) on Apple Silicon using MLX. It supports local inference, fine-tuning via LoRA\u002FDoRA, and embeddings generation.\n\n### When to Use This Skill\n- Running LLM\u002FVLM inference on macOS\u002FiOS with Apple Silicon\n- Streaming text generation from local models\n- Vision tasks with images\u002Fvideo (VLMs)\n- Tool calling \u002F function calling with models\n- LoRA adapter training and fine-tuning\n- Text embeddings for RAG\u002Fsemantic search\n\n### Architecture Overview\n```\nMLXLMCommon     - Core infrastructure (ModelContainer, ChatSession, KVCache, etc.)\nMLXLLM          - Text-only LLM support (Llama, Qwen, Gemma, Phi, DeepSeek, etc. - examples, not exhaustive)\nMLXVLM          - Vision-Language Models (Qwen2-VL, PaliGemma, Gemma3, etc. - examples, not exhaustive)\nEmbedders       - Embedding models (BGE, Nomic, MiniLM)\n```\n\n## 2. Key File Reference\n\n| Purpose | File Path |\n|---------|-----------|\n| Thread-safe model wrapper | `Libraries\u002FMLXLMCommon\u002FModelContainer.swift` |\n| Simplified chat API | `Libraries\u002FMLXLMCommon\u002FChatSession.swift` |\n| Generation & streaming | `Libraries\u002FMLXLMCommon\u002FEvaluate.swift` |\n| KV cache types | `Libraries\u002FMLXLMCommon\u002FKVCache.swift` |\n| Model configuration | `Libraries\u002FMLXLMCommon\u002FModelConfiguration.swift` |\n| Chat message types | `Libraries\u002FMLXLMCommon\u002FChat.swift` |\n| Tool call processing | `Libraries\u002FMLXLMCommon\u002FTool\u002FToolCallFormat.swift` |\n| Concurrency utilities | `Libraries\u002FMLXLMCommon\u002FUtilities\u002FSerialAccessContainer.swift` |\n| LLM factory & registry | `Libraries\u002FMLXLLM\u002FLLMModelFactory.swift` |\n| VLM factory & registry | `Libraries\u002FMLXVLM\u002FVLMModelFactory.swift` |\n| LoRA configuration | `Libraries\u002FMLXLMCommon\u002FAdapters\u002FLoRA\u002FLoRAContainer.swift` |\n| LoRA training | `Libraries\u002FMLXLLM\u002FLoraTrain.swift` |\n\n## 3. Quick Start\n\n### LLM Chat (Simplest API)\n\n```swift\nimport MLXLLM\nimport MLXLMCommon\n\n\u002F\u002F Load model (downloads from HuggingFace automatically)\nlet modelContainer = try await LLMModelFactory.shared.loadContainer(\n    configuration: .init(id: \"mlx-community\u002FQwen3-4B-4bit\")\n)\n\n\u002F\u002F Create chat session\nlet session = ChatSession(modelContainer)\n\n\u002F\u002F Single response\nlet response = try await session.respond(to: \"What is Swift?\")\nprint(response)\n\n\u002F\u002F Streaming response\nfor try await chunk in session.streamResponse(to: \"Explain concurrency\") {\n    print(chunk, terminator: \"\")\n}\n```\n\n### VLM with Image\n\n```swift\nimport MLXVLM\nimport MLXLMCommon\n\nlet modelContainer = try await VLMModelFactory.shared.loadContainer(\n    configuration: .init(id: \"mlx-community\u002FQwen2-VL-2B-Instruct-4bit\")\n)\n\nlet session = ChatSession(modelContainer)\n\n\u002F\u002F With image (video is also an optional parameter)\nlet image = UserInput.Image.url(imageURL)\nlet response = try await session.respond(\n    to: \"Describe this image\",\n    image: image,\n    video: nil  \u002F\u002F Optional video parameter\n)\n```\n\n### Embeddings\n\n```swift\nimport Embedders\n\n\u002F\u002F Note: Embedders uses loadModelContainer() helper (not a factory pattern)\nlet container = try await loadModelContainer(\n    configuration: ModelConfiguration(id: \"mlx-community\u002Fbge-small-en-v1.5-mlx\")\n)\n\nlet embeddings = await container.perform { model, tokenizer, pooler in\n    let tokens = tokenizer.encode(text: \"Hello world\")\n    let input = MLXArray(tokens).expandedDimensions(axis: 0)\n    let output = model(input)\n    let pooled = pooler(output, normalize: true)\n    eval(pooled)\n    return pooled\n}\n```\n\n## 4. Primary Workflow: LLM Inference\n\n### ChatSession API (Recommended)\n\n`ChatSession` manages conversation history and KV cache automatically:\n\n```swift\nlet session = ChatSession(\n    modelContainer,\n    instructions: \"You are a helpful assistant\",  \u002F\u002F System prompt\n    generateParameters: GenerateParameters(\n        maxTokens: 500,\n        temperature: 0.7\n    )\n)\n\n\u002F\u002F Multi-turn conversation (history preserved automatically)\nlet r1 = try await session.respond(to: \"What is 2+2?\")\nlet r2 = try await session.respond(to: \"And if you multiply that by 3?\")\n\n\u002F\u002F Clear session to start fresh\nawait session.clear()\n```\n\n### Streaming with generate()\n\nFor lower-level control, use `generate()` directly:\n\n```swift\nlet input = try await modelContainer.prepare(input: UserInput(prompt: .text(\"Hello\")))\nlet stream = try await modelContainer.generate(input: input, parameters: GenerateParameters())\n\nfor await generation in stream {\n    switch generation {\n    case .chunk(let text):\n        print(text, terminator: \"\")\n    case .info(let info):\n        print(\"\\n\\(info.tokensPerSecond) tok\u002Fs\")\n    case .toolCall(let call):\n        \u002F\u002F Handle tool call\n        break\n    }\n}\n```\n\n### Tool Calling\n\n```swift\n\u002F\u002F 1. Define tool\nstruct WeatherInput: Codable { let location: String }\nstruct WeatherOutput: Codable { let temperature: Double; let conditions: String }\n\nlet weatherTool = Tool\u003CWeatherInput, WeatherOutput>(\n    name: \"get_weather\",\n    description: \"Get current weather\",\n    parameters: [.required(\"location\", type: .string, description: \"City name\")]\n) { input in\n    WeatherOutput(temperature: 22.0, conditions: \"Sunny\")\n}\n\n\u002F\u002F 2. Include tool schema in request\nlet input = UserInput(\n    prompt: .text(\"What's the weather in Tokyo?\"),\n    tools: [weatherTool.schema]\n)\n\n\u002F\u002F 3. Handle tool calls in generation stream\nfor await generation in try await modelContainer.generate(input: input, parameters: params) {\n    switch generation {\n    case .chunk(let text): print(text)\n    case .toolCall(let call):\n        let result = try await call.execute(with: weatherTool)\n        print(\"Weather: \\(result.conditions)\")\n    case .info: break\n    }\n}\n```\n\nSee [references\u002Ftool-calling.md](references\u002Ftool-calling.md) for multi-turn and feeding results back.\n\n### GenerateParameters\n\n```swift\nlet params = GenerateParameters(\n    maxTokens: 1000,           \u002F\u002F nil = unlimited\n    maxKVSize: 4096,           \u002F\u002F Sliding window (uses RotatingKVCache)\n    kvBits: 4,                 \u002F\u002F Quantized cache (4 or 8 bit)\n    temperature: 0.7,          \u002F\u002F 0 = greedy\u002Fargmax\n    topP: 0.9,                 \u002F\u002F Nucleus sampling\n    repetitionPenalty: 1.1,    \u002F\u002F Penalize repeats\n    repetitionContextSize: 20  \u002F\u002F Window for penalty\n)\n```\n\n### Prompt Caching \u002F History Re-hydration\n\nRestore chat from persisted history:\n\n```swift\nlet history: [Chat.Message] = [\n    .system(\"You are helpful\"),\n    .user(\"Hello\"),\n    .assistant(\"Hi there!\")\n]\n\nlet session = ChatSession(\n    modelContainer,\n    history: history\n)\n\u002F\u002F Continues from this point\n```\n\n## 5. Secondary Workflow: VLM Inference\n\n### Image Input Types\n\n```swift\n\u002F\u002F From URL (file or remote)\nlet image = UserInput.Image.url(fileURL)\n\n\u002F\u002F From CIImage\nlet image = UserInput.Image.ciImage(ciImage)\n\n\u002F\u002F From MLXArray directly\nlet image = UserInput.Image.array(mlxArray)\n```\n\n### Video Input\n\n```swift\n\u002F\u002F From URL (file or remote)\nlet video = UserInput.Video.url(videoURL)\n\n\u002F\u002F From AVFoundation asset\nlet video = UserInput.Video.avAsset(avAsset)\n\n\u002F\u002F From pre-extracted frames\nlet video = UserInput.Video.frames(videoFrames)\n\nlet response = try await session.respond(\n    to: \"What happens in this video?\",\n    video: video\n)\n```\n\n### Multiple Images\n\n```swift\nlet images: [UserInput.Image] = [\n    .url(url1),\n    .url(url2)\n]\n\nlet response = try await session.respond(\n    to: \"Compare these two images\",\n    images: images,\n    videos: []\n)\n```\n\n### VLM-Specific Processing\n\n```swift\nlet session = ChatSession(\n    modelContainer,\n    processing: UserInput.Processing(\n        resize: CGSize(width: 512, height: 512)  \u002F\u002F Resize images\n    )\n)\n```\n\n## 6. Best Practices\n\n### DO\n\n```swift\n\u002F\u002F DO: Use ChatSession for multi-turn conversations\nlet session = ChatSession(modelContainer)\n\n\u002F\u002F DO: Use AsyncStream APIs (modern, Swift concurrency)\nfor try await chunk in session.streamResponse(to: prompt) { ... }\n\n\u002F\u002F DO: Check Task.isCancelled in long-running loops\nfor try await generation in stream {\n    if Task.isCancelled { break }\n    \u002F\u002F process generation\n}\n\n\u002F\u002F DO: Use ModelContainer.perform() for thread-safe access\nawait modelContainer.perform { context in\n    \u002F\u002F Access model, tokenizer safely\n    let tokens = try context.tokenizer.applyChatTemplate(messages: messages)\n    return tokens\n}\n\n\u002F\u002F DO: When breaking early from generation, use generateTask() to get a task handle\n\u002F\u002F This is the lower-level API used internally by ChatSession\nlet (stream, task) = generateTask(...)  \u002F\u002F Returns (AsyncStream, Task)\n\nfor await item in stream {\n    if shouldStop { break }\n}\nawait task.value  \u002F\u002F Ensures KV cache cleanup before next generation\n```\n\n> `generateTask()` is defined in `Evaluate.swift`. Most users should use `ChatSession` which handles this internally.\n\n### DON'T\n\n```swift\n\u002F\u002F DON'T: Share MLXArray across tasks (not Sendable)\nlet array = MLXArray(...)\nTask { array.sum() }  \u002F\u002F Wrong!\n\n\u002F\u002F DON'T: Use deprecated callback-based generation\n\u002F\u002F Old:\ngenerate(input: input, parameters: params) { tokens in ... }  \u002F\u002F Deprecated\n\u002F\u002F New:\nfor await generation in try generate(input: input, parameters: params, context: context) { ... }\n\n\u002F\u002F DON'T: Use old perform(model, tokenizer) signature\n\u002F\u002F Old:\nmodelContainer.perform { model, tokenizer in ... }  \u002F\u002F Deprecated\n\u002F\u002F New:\nmodelContainer.perform { context in ... }\n\n\u002F\u002F DON'T: Forget to eval() MLXArrays before returning from perform()\nawait modelContainer.perform { context in\n    let result = context.model(input)\n    eval(result)  \u002F\u002F Required before returning\n    return result.item(Float.self)\n}\n```\n\n### Thread Safety\n\n- `ModelContainer` is `Sendable` and thread-safe\n- `ChatSession` is NOT thread-safe (use from single task)\n- `MLXArray` is NOT `Sendable` - don't pass across isolation boundaries\n- Use `SendableBox` for transferring non-Sendable data in consuming contexts\n\n### Memory Management\n\n```swift\n\u002F\u002F For long contexts, use sliding window cache\nlet params = GenerateParameters(maxKVSize: 4096)\n\n\u002F\u002F For memory efficiency, use quantized cache\nlet params = GenerateParameters(kvBits: 4)  \u002F\u002F or 8\n\n\u002F\u002F Clear session cache when done\nawait session.clear()\n```\n\n## 7. Reference Links\n\nFor detailed documentation on specific topics, see:\n\n| Reference | When to Use |\n|-----------|-------------|\n| [references\u002Fmodel-container.md](references\u002Fmodel-container.md) | Loading models, ModelContainer API, ModelConfiguration |\n| [references\u002Fkv-cache.md](references\u002Fkv-cache.md) | Cache types, memory optimization, cache serialization |\n| [references\u002Fconcurrency.md](references\u002Fconcurrency.md) | Thread safety, SerialAccessContainer, async patterns |\n| [references\u002Ftool-calling.md](references\u002Ftool-calling.md) | Function calling, tool formats, ToolCallProcessor |\n| [references\u002Ftokenizer-chat.md](references\u002Ftokenizer-chat.md) | Tokenizer, Chat.Message, EOS tokens |\n| [references\u002Fsupported-models.md](references\u002Fsupported-models.md) | Model families, registries, model-specific config |\n| [references\u002Flora-adapters.md](references\u002Flora-adapters.md) | LoRA\u002FDoRA\u002FQLoRA, loading adapters |\n| [references\u002Ftraining.md](references\u002Ftraining.md) | LoRATrain API, fine-tuning |\n| [references\u002Fembeddings.md](references\u002Fembeddings.md) | EmbeddingModel, pooling, use cases |\n\n## 8. Deprecated Patterns Summary\n\nMost common migrations (see individual reference files for topic-specific deprecations):\n\n| If you see... | Use instead... |\n|---------------|----------------|\n| `generate(... didGenerate:)` callback | `generate(...) -> AsyncStream` |\n| `perform { model, tokenizer in }` | `perform { context in }` |\n| `TokenIterator(prompt: MLXArray)` | `TokenIterator(input: LMInput)` |\n| `ModelRegistry` typealias | `LLMRegistry` or `VLMRegistry` |\n| `createAttentionMask(h:cache:[KVCache]?)` | `createAttentionMask(h:cache:KVCache?)` |\n\nEach reference file contains a \"Deprecated Patterns\" section with topic-specific migrations.\n\n## 9. Automatic vs Manual Configuration\n\n### Automatic Behaviors (NO developer action needed)\n\nThe framework handles these automatically:\n\n| Feature | Details |\n|---------|---------|\n| **EOS token loading** | Loaded from `config.json` |\n| **EOS token override** | Priority: `generation_config.json` > `config.json` > defaults |\n| **EOS token merging** | All sources merged at generation time |\n| **EOS token detection** | Stops generation automatically when EOS encountered |\n| **Chat template application** | Applied automatically via `applyChatTemplate()` |\n| **Tool call format detection** | Inferred from `model_type` in `config.json` |\n| **Cache type selection** | Based on GenerateParameters (`maxKVSize`, `kvBits`) |\n| **Tokenizer loading** | Loaded from `tokenizer.json` automatically |\n| **Model weights loading** | Downloaded and loaded from HuggingFace |\n\n### Optional Configuration (Developer MAY configure)\n\n| Feature | When to Configure |\n|---------|-------------------|\n| `extraEOSTokens` | Only if model has unlisted stop tokens |\n| `toolCallFormat` | Only to override auto-detection |\n| `maxKVSize` | To enable sliding window cache |\n| `kvBits` | To enable quantized cache (4 or 8 bit) |\n| `maxTokens` | To limit output length |\n",{"name":5,"triggers":48,"description":7},[26,49,50,51,52,53,54],"mlx-swift","mlx-lm","apple silicon llm","local llm swift","vision language model swift","lora training swift","main","f4cb2d49f7667e79ef7804f50396d5aaffdf1f56","人工智能与大型语言模型\u002Fmlx-swift-lm",1,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification","https:\u002F\u002Fgithub.com\u002FKernel8901","a little little little dobe in life but science geek","2026-04-05 16:43:56",{"source":8,"skillName":5,"commands":64,"prompt":74,"zipAvailable":75},[65,68,71],{"tool":66,"command":67},"npx","npx skills add Kernel8901\u002Fai-agent-skills-classification --skill swift-mlx-lm",{"tool":69,"command":70},"bunx","bunx skills add Kernel8901\u002Fai-agent-skills-classification --skill swift-mlx-lm",{"tool":72,"command":73},"pnpm","pnpm dlx skills add Kernel8901\u002Fai-agent-skills-classification --skill swift-mlx-lm","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Fskill.md",true,[77,83,87,91,95,99,103,107,111,115],{"path":78,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":81,"sourceHtmlUrl":82},"_meta.json","REFERENCE","参考文档",289,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002F_meta.json",{"path":84,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":85,"sourceHtmlUrl":86},"references\u002Fconcurrency.md",7571,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Freferences\u002Fconcurrency.md",{"path":88,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":89,"sourceHtmlUrl":90},"references\u002Fembeddings.md",7889,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Freferences\u002Fembeddings.md",{"path":92,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":93,"sourceHtmlUrl":94},"references\u002Fkv-cache.md",7266,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Freferences\u002Fkv-cache.md",{"path":96,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":97,"sourceHtmlUrl":98},"references\u002Flora-adapters.md",7693,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Freferences\u002Flora-adapters.md",{"path":100,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":101,"sourceHtmlUrl":102},"references\u002Fmodel-container.md",7467,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Freferences\u002Fmodel-container.md",{"path":104,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":105,"sourceHtmlUrl":106},"references\u002Fsupported-models.md",7272,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Freferences\u002Fsupported-models.md",{"path":108,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":109,"sourceHtmlUrl":110},"references\u002Ftokenizer-chat.md",7561,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Freferences\u002Ftokenizer-chat.md",{"path":112,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":113,"sourceHtmlUrl":114},"references\u002Ftool-calling.md",9057,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Freferences\u002Ftool-calling.md",{"path":116,"resourceType":79,"resourceTypeDesc":80,"sizeBytes":117,"sourceHtmlUrl":118},"references\u002Ftraining.md",8970,"https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F人工智能与大型语言模型\u002Fmlx-swift-lm\u002Freferences\u002Ftraining.md",[120,144,176,187,197,227],{"skillNo":121,"name":122,"nameZh":6,"description":123,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":126,"publishedAt":43},"01M6Q5R6WZ06RRB9RQTW54B34B","zyfai","Earn yield on any Ethereum wallet on Base, Arbitrum, and Plasma. Use when a user wants passive DeFi yield on their funds. Deploys a non-custodial subaccount (Safe) linked to their EOA, enables automated yield optimization, and lets them deposit\u002Fwithdraw anytime.","https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F金融\u002Fzyfai-sdk\u002FSKILL.md",[],[122,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143],"safe","ethereum","defi","Optimization","user","use","Wallet","base","deposit","withdraw","yield","arbitrum","non-custodial","passive","defi yield","earn","yield optimization",{"skillNo":145,"name":146,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":147,"categories":148,"keywords":158,"publishedAt":43},"01M6Q5R337B5JSNER4P4YS03TN","zerion-api","https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F金融\u002Fzerion-api\u002FSKILL.md",[149,152,153],{"categoryNo":150,"categoryName":151,"taxonomyType":20,"taxonomyTypeDesc":21},"01JBSEEDCAT000000000000010","后端与接口",{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":154,"categoryName":155,"taxonomyType":156,"taxonomyTypeDesc":157},"01JBSEEDCAT000000000000030","开发工程师","PROFESSION","适用职业",[159,160,161,162,163,164,165,129,166,133,167,168,169,170,171,172,173,174,175],"api","zerion api","zerion","MCP","mcp server","server","blockchain","transactions","crypto wallet","crypto","evm","solana","query","across","positions","token","prices",{"skillNo":177,"name":178,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":179,"categories":180,"keywords":184,"publishedAt":43},"01M6Q5QZFYTDAZMZ27BN5VZXAR","zerion-api-skill","https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F金融\u002Fzerion-api-skill\u002FSKILL.md",[181,182,183],{"categoryNo":150,"categoryName":151,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":154,"categoryName":155,"taxonomyType":156,"taxonomyTypeDesc":157},[159,185,186,160,161,162,163,164,165,129,166,133,168,169,170,171,172,173,174,175],"api skill","zerion api skill",{"skillNo":188,"name":189,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":190,"categories":191,"keywords":195,"publishedAt":43},"01M6Q5QW17QW98RA3FE45E4RD8","zerion-api-mcp","https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F金融\u002Fzerion-api-mcp\u002FSKILL.md",[192,193,194],{"categoryNo":150,"categoryName":151,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":18,"categoryName":19,"taxonomyType":20,"taxonomyTypeDesc":21},{"categoryNo":154,"categoryName":155,"taxonomyType":156,"taxonomyTypeDesc":157},[162,159,196,160,161,163,164,129,166,133,167,168,169,170,171,172,173,174,175],"zerion api mcp",{"skillNo":198,"name":199,"nameZh":6,"description":200,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":201,"categories":202,"keywords":206,"publishedAt":43},"01M6Q5QQYZB7A501BDEQWKGSJT","yahoo-finance-forex","Fetches real-time FOREX news and market data from Yahoo Finance for major currency pairs (EUR\u002FUSD, GBP\u002FUSD, USD\u002FJPY, etc.). Analyzes sentiment and provides trading context.","https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F金融\u002Fyahoo-finance-forex\u002FSKILL.md",[203],{"categoryNo":204,"categoryName":205,"taxonomyType":156,"taxonomyTypeDesc":157},"01JBSEEDCAT000000000000040","财务",[207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226],"finance","yahoo finance","forex","yahoo","yahoo finance forex","github","https","sentiment","real-time","python3","install","Market","usd","news","time","currency","gbp","pip","market data","for",{"skillNo":228,"name":229,"nameZh":6,"description":230,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":6,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":231,"categories":232,"keywords":234,"publishedAt":43},"01M6Q5QM6VS84WZVM7BKXXPF32","yahoo-finance","This skill should be used when the user asks to \"get stock prices\", \"check stock quotes\", \"look up earnings\", \"get financial data\", \"find trending stocks\", or needs stock market data from Yahoo Finance.","https:\u002F\u002Fgithub.com\u002FKernel8901\u002Fai-agent-skills-classification\u002Fblob\u002Fmain\u002F金融\u002Fyahoo-finance-cli\u002FSKILL.md",[233],{"categoryNo":204,"categoryName":205,"taxonomyType":156,"taxonomyTypeDesc":157},[207,208,210,235,236,237,238,239,240,218,30,241,242,243,244,245,246,247,175,225],"cli","homebrew","find","check","brew","up","get","stock","link","trending","sf","quotes","ln"]