[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6XEFS2HBYW4JAXZPW2HDAZ4":3,"skill-same:01M6XEFS2HBYW4JAXZPW2HDAZ4":70},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":43,"commitSha":44,"skillDirectory":45,"forksCount":13,"repositoryHtmlUrl":46,"repositoryHomepageUrl":6,"ownerHtmlUrl":47,"ownerBlogUrl":6,"ownerDescription":48,"updatedAt":49,"install":50,"resources":63},"01M6XEFS2HBYW4JAXZPW2HDAZ4","python-parallelization",null,"Transform sequential Python code into parallel\u002Fconcurrent implementations. Use when asked to parallelize Python code, improve code performance through concurrency, convert loops to parallel execution, or identify parallelization opportunities. Handles CPU-bound (multiprocessing), I\u002FO-bound (asyncio, threading), and data-parallel (vectorization) scenarios.","Waaaalnut\u002Fskillsbench","Waaaalnut","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F103995656?v=4","01M6XCS9K1V51EY3JMB7DGMX24","01M6XCS9Q4WPW88W62C55G1H44",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench\u002Fblob\u002Fmain\u002Ftasks\u002Fparallel-tfidf-search\u002Fenvironment\u002Fskills\u002Fpython-parallelization\u002FSKILL.md",[],[18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37],"python","python parallelization","parallelization","when","python code","threading","loops","CPU","transform","parallel execution","use","parallel","convert","improve","concurrency","execution","data","vectorization","code","performance","2026-11-10 10:12:30",false,"---\nname: python-parallelization\ndescription: Transform sequential Python code into parallel\u002Fconcurrent implementations. Use when asked to parallelize Python code, improve code performance through concurrency, convert loops to parallel execution, or identify parallelization opportunities. Handles CPU-bound (multiprocessing), I\u002FO-bound (asyncio, threading), and data-parallel (vectorization) scenarios.\n---\n\n# Python Parallelization Skill\n\nTransform sequential Python code to leverage parallel and concurrent execution patterns.\n\n## Workflow\n\n1. **Analyze** the code to identify parallelization candidates\n2. **Classify** the workload type (CPU-bound, I\u002FO-bound, or data-parallel)\n3. **Select** the appropriate parallelization strategy\n4. **Transform** the code with proper synchronization and error handling\n5. **Verify** correctness and measure expected speedup\n\n## Parallelization Decision Tree\n\n```\nIs the bottleneck CPU-bound or I\u002FO-bound?\n\nCPU-bound (computation-heavy):\n├── Independent iterations? → multiprocessing.Pool \u002F ProcessPoolExecutor\n├── Shared state needed? → multiprocessing with Manager or shared memory\n├── NumPy\u002FPandas operations? → Vectorization first, then consider numba\u002Fdask\n└── Large data chunks? → chunked processing with Pool.map\n\nI\u002FO-bound (network, disk, database):\n├── Many independent requests? → asyncio with aiohttp\u002Faiofiles\n├── Legacy sync code? → ThreadPoolExecutor\n├── Mixed sync\u002Fasync? → asyncio.to_thread()\n└── Database queries? → Connection pooling + async drivers\n\nData-parallel (array\u002Fmatrix ops):\n├── NumPy arrays? → Vectorize, avoid Python loops\n├── Pandas DataFrames? → Use built-in vectorized methods\n├── Large datasets? → Dask for out-of-core parallelism\n└── GPU available? → Consider CuPy or JAX\n```\n\n## Transformation Patterns\n\n### Pattern 1: Loop to ProcessPoolExecutor (CPU-bound)\n\n**Before:**\n```python\nresults = []\nfor item in items:\n    results.append(expensive_computation(item))\n```\n\n**After:**\n```python\nfrom concurrent.futures import ProcessPoolExecutor\n\nwith ProcessPoolExecutor() as executor:\n    results = list(executor.map(expensive_computation, items))\n```\n\n### Pattern 2: Sequential I\u002FO to Async (I\u002FO-bound)\n\n**Before:**\n```python\nimport requests\n\ndef fetch_all(urls):\n    return [requests.get(url).json() for url in urls]\n```\n\n**After:**\n```python\nimport asyncio\nimport aiohttp\n\nasync def fetch_all(urls):\n    async with aiohttp.ClientSession() as session:\n        tasks = [fetch_one(session, url) for url in urls]\n        return await asyncio.gather(*tasks)\n\nasync def fetch_one(session, url):\n    async with session.get(url) as response:\n        return await response.json()\n```\n\n### Pattern 3: Nested Loops to Vectorization\n\n**Before:**\n```python\nresult = []\nfor i in range(len(a)):\n    row = []\n    for j in range(len(b)):\n        row.append(a[i] * b[j])\n    result.append(row)\n```\n\n**After:**\n```python\nimport numpy as np\nresult = np.outer(a, b)\n```\n\n### Pattern 4: Mixed CPU\u002FIO with asyncio\n\n```python\nimport asyncio\nfrom concurrent.futures import ProcessPoolExecutor\n\nasync def hybrid_pipeline(data, urls):\n    loop = asyncio.get_event_loop()\n\n    # CPU-bound in process pool\n    with ProcessPoolExecutor() as pool:\n        processed = await loop.run_in_executor(pool, cpu_heavy_fn, data)\n\n    # I\u002FO-bound with async\n    results = await asyncio.gather(*[fetch(url) for url in urls])\n\n    return processed, results\n```\n\n## Parallelization Candidates\n\nLook for these patterns in code:\n\n| Pattern | Indicator | Strategy |\n|---------|-----------|----------|\n| `for item in collection` with independent iterations | No shared mutation | `Pool.map` \u002F `executor.map` |\n| Multiple `requests.get()` or file reads | Sequential I\u002FO | `asyncio.gather()` |\n| Nested loops over arrays | Numerical computation | NumPy vectorization |\n| `time.sleep()` or blocking waits | Waiting on external | Threading or async |\n| Large list comprehensions | Independent transforms | `Pool.map` with chunking |\n\n## Safety Requirements\n\nAlways preserve correctness when parallelizing:\n\n1. **Identify shared state** - variables modified across iterations break parallelism\n2. **Check dependencies** - iteration N depending on N-1 requires sequential execution\n3. **Handle exceptions** - wrap parallel code in try\u002Fexcept, use `executor.submit()` for granular error handling\n4. **Manage resources** - use context managers, limit worker count to avoid exhaustion\n5. **Preserve ordering** - use `map()` over `submit()` when order matters\n\n## Common Pitfalls\n\n- **GIL trap**: Threading doesn't help CPU-bound Python code—use multiprocessing\n- **Pickle failures**: Lambda functions and nested classes can't be pickled for multiprocessing\n- **Memory explosion**: ProcessPoolExecutor copies data to each process—use shared memory for large data\n- **Async in sync**: Can't just add `async` to existing code—requires restructuring call chain\n- **Over-parallelization**: Parallel overhead exceeds gains for small workloads (\u003C1000 items typically)\n\n## Verification Checklist\n\nBefore finalizing transformed code:\n\n- [ ] Output matches sequential version for test inputs\n- [ ] No race conditions (shared mutable state properly synchronized)\n- [ ] Exceptions are caught and handled appropriately\n- [ ] Resources are properly cleaned up (pools closed, connections released)\n- [ ] Worker count is bounded (default or explicit limit)\n- [ ] Added appropriate imports\n","# Python Parallelization Skill\n\nTransform sequential Python code to leverage parallel and concurrent execution patterns.\n\n## Workflow\n\n1. **Analyze** the code to identify parallelization candidates\n2. **Classify** the workload type (CPU-bound, I\u002FO-bound, or data-parallel)\n3. **Select** the appropriate parallelization strategy\n4. **Transform** the code with proper synchronization and error handling\n5. **Verify** correctness and measure expected speedup\n\n## Parallelization Decision Tree\n\n```\nIs the bottleneck CPU-bound or I\u002FO-bound?\n\nCPU-bound (computation-heavy):\n├── Independent iterations? → multiprocessing.Pool \u002F ProcessPoolExecutor\n├── Shared state needed? → multiprocessing with Manager or shared memory\n├── NumPy\u002FPandas operations? → Vectorization first, then consider numba\u002Fdask\n└── Large data chunks? → chunked processing with Pool.map\n\nI\u002FO-bound (network, disk, database):\n├── Many independent requests? → asyncio with aiohttp\u002Faiofiles\n├── Legacy sync code? → ThreadPoolExecutor\n├── Mixed sync\u002Fasync? → asyncio.to_thread()\n└── Database queries? → Connection pooling + async drivers\n\nData-parallel (array\u002Fmatrix ops):\n├── NumPy arrays? → Vectorize, avoid Python loops\n├── Pandas DataFrames? → Use built-in vectorized methods\n├── Large datasets? → Dask for out-of-core parallelism\n└── GPU available? → Consider CuPy or JAX\n```\n\n## Transformation Patterns\n\n### Pattern 1: Loop to ProcessPoolExecutor (CPU-bound)\n\n**Before:**\n```python\nresults = []\nfor item in items:\n    results.append(expensive_computation(item))\n```\n\n**After:**\n```python\nfrom concurrent.futures import ProcessPoolExecutor\n\nwith ProcessPoolExecutor() as executor:\n    results = list(executor.map(expensive_computation, items))\n```\n\n### Pattern 2: Sequential I\u002FO to Async (I\u002FO-bound)\n\n**Before:**\n```python\nimport requests\n\ndef fetch_all(urls):\n    return [requests.get(url).json() for url in urls]\n```\n\n**After:**\n```python\nimport asyncio\nimport aiohttp\n\nasync def fetch_all(urls):\n    async with aiohttp.ClientSession() as session:\n        tasks = [fetch_one(session, url) for url in urls]\n        return await asyncio.gather(*tasks)\n\nasync def fetch_one(session, url):\n    async with session.get(url) as response:\n        return await response.json()\n```\n\n### Pattern 3: Nested Loops to Vectorization\n\n**Before:**\n```python\nresult = []\nfor i in range(len(a)):\n    row = []\n    for j in range(len(b)):\n        row.append(a[i] * b[j])\n    result.append(row)\n```\n\n**After:**\n```python\nimport numpy as np\nresult = np.outer(a, b)\n```\n\n### Pattern 4: Mixed CPU\u002FIO with asyncio\n\n```python\nimport asyncio\nfrom concurrent.futures import ProcessPoolExecutor\n\nasync def hybrid_pipeline(data, urls):\n    loop = asyncio.get_event_loop()\n\n    # CPU-bound in process pool\n    with ProcessPoolExecutor() as pool:\n        processed = await loop.run_in_executor(pool, cpu_heavy_fn, data)\n\n    # I\u002FO-bound with async\n    results = await asyncio.gather(*[fetch(url) for url in urls])\n\n    return processed, results\n```\n\n## Parallelization Candidates\n\nLook for these patterns in code:\n\n| Pattern | Indicator | Strategy |\n|---------|-----------|----------|\n| `for item in collection` with independent iterations | No shared mutation | `Pool.map` \u002F `executor.map` |\n| Multiple `requests.get()` or file reads | Sequential I\u002FO | `asyncio.gather()` |\n| Nested loops over arrays | Numerical computation | NumPy vectorization |\n| `time.sleep()` or blocking waits | Waiting on external | Threading or async |\n| Large list comprehensions | Independent transforms | `Pool.map` with chunking |\n\n## Safety Requirements\n\nAlways preserve correctness when parallelizing:\n\n1. **Identify shared state** - variables modified across iterations break parallelism\n2. **Check dependencies** - iteration N depending on N-1 requires sequential execution\n3. **Handle exceptions** - wrap parallel code in try\u002Fexcept, use `executor.submit()` for granular error handling\n4. **Manage resources** - use context managers, limit worker count to avoid exhaustion\n5. **Preserve ordering** - use `map()` over `submit()` when order matters\n\n## Common Pitfalls\n\n- **GIL trap**: Threading doesn't help CPU-bound Python code—use multiprocessing\n- **Pickle failures**: Lambda functions and nested classes can't be pickled for multiprocessing\n- **Memory explosion**: ProcessPoolExecutor copies data to each process—use shared memory for large data\n- **Async in sync**: Can't just add `async` to existing code—requires restructuring call chain\n- **Over-parallelization**: Parallel overhead exceeds gains for small workloads (\u003C1000 items typically)\n\n## Verification Checklist\n\nBefore finalizing transformed code:\n\n- [ ] Output matches sequential version for test inputs\n- [ ] No race conditions (shared mutable state properly synchronized)\n- [ ] Exceptions are caught and handled appropriately\n- [ ] Resources are properly cleaned up (pools closed, connections released)\n- [ ] Worker count is bounded (default or explicit limit)\n- [ ] Added appropriate imports\n",{"name":5,"description":7},"main","7eeb258ada72b7f58119b0e7a2c00ea89d9beedd","tasks\u002Fparallel-tfidf-search\u002Fenvironment\u002Fskills\u002Fpython-parallelization","https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench","https:\u002F\u002Fgithub.com\u002FWaaaalnut","student of Zhejiang University","2026-04-21 23:16:23",{"source":8,"skillName":5,"commands":51,"prompt":61,"zipAvailable":62},[52,55,58],{"tool":53,"command":54},"npx","npx skills add Waaaalnut\u002Fskillsbench --skill python-parallelization",{"tool":56,"command":57},"bunx","bunx skills add Waaaalnut\u002Fskillsbench --skill python-parallelization",{"tool":59,"command":60},"pnpm","pnpm dlx skills add Waaaalnut\u002Fskillsbench --skill python-parallelization","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench\u002Fblob\u002Fmain\u002Ftasks\u002Fparallel-tfidf-search\u002Fenvironment\u002Fskills\u002Fpython-parallelization\u002FSKILL.md",true,[64],{"path":65,"resourceType":66,"resourceTypeDesc":67,"sizeBytes":68,"sourceHtmlUrl":69},"references\u002Fadvanced_techniques.md","REFERENCE","参考文档",5508,"https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench\u002Fblob\u002Fmain\u002Ftasks\u002Fparallel-tfidf-search\u002Fenvironment\u002Fskills\u002Fpython-parallelization\u002Freferences\u002Fadvanced_techniques.md",[71,84,99,125,147,166],{"skillNo":72,"name":73,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":81,"publishedAt":38},"01M6XEWMJ6VY3ZC53MPDCPG3GH","data-reconciliation","https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench\u002Fblob\u002Fmain\u002Ftasks\u002Fxlsx-recover-data\u002Fenvironment\u002Fskills\u002Fdata-reconciliation\u002FSKILL.md",[76],{"categoryNo":77,"categoryName":78,"taxonomyType":79,"taxonomyTypeDesc":80},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[82,83,34],"data reconciliation","reconciliation",{"skillNo":85,"name":86,"nameZh":6,"description":87,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":88,"categories":89,"keywords":90,"publishedAt":38},"01M6XEW8MNPMF1KRXV6ATGFRQF","speech-to-text","Transcribe video to timestamped text using Whisper tiny model (pre-installed).","https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench\u002Fblob\u002Fmain\u002Ftasks\u002Fvideo-tutorial-indexer\u002Fenvironment\u002Fskills\u002Fspeech-to-text\u002FSKILL.md",[],[91,92,93,94,95,96,97,98],"speech","text","speech to text","transcribe video","whisper","model","Video","transcribe",{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":110,"publishedAt":38},"01M6XEW3FQZP0T1RKQQJFXVY9T","video-processor","Process videos by removing segments and concatenating remaining parts. Use when you need to remove detected pauses\u002Fopenings from videos, create highlight reels, or batch process segment removals using ffmpeg filter_complex.","https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench\u002Fblob\u002Fmain\u002Ftasks\u002Fvideo-silence-remover\u002Fenvironment\u002Fskills\u002Fvideo-processor\u002FSKILL.md",[105],{"categoryNo":106,"categoryName":107,"taxonomyType":108,"taxonomyTypeDesc":109},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",[97,111,112,113,114,115,21,116,117,118,119,120,121,122,123,28,124],"processor","video processor","batch","create","reels","parts","segments","process","videos","you","segment","filter","ffmpeg","remove",{"skillNo":126,"name":127,"nameZh":6,"description":128,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":129,"categories":130,"keywords":132,"publishedAt":38},"01M6XEVXJ5KZ0NTY3XZKRY4AS1","silence-detector","Detect initial silence segments in audio\u002Fvideo using energy-based analysis. Use when you need to find low-energy periods at the start of recordings (title slides, setup time, pre-roll silence).","https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench\u002Fblob\u002Fmain\u002Ftasks\u002Fvideo-silence-remover\u002Fenvironment\u002Fskills\u002Fsilence-detector\u002FSKILL.md",[131],{"categoryNo":106,"categoryName":107,"taxonomyType":108,"taxonomyTypeDesc":109},[133,134,135,136,21,120,137,138,139,140,141,117,142,143,97,144,145,146,28],"detector","silence detector","title","slides","time","energy","at","recordings","detect","find","Audio","analysis","start","setup",{"skillNo":148,"name":149,"nameZh":6,"description":150,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":151,"categories":152,"keywords":153,"publishedAt":38},"01M6XEVT3NGRN1FT7JPMAPXS6M","segment-combiner","Combine multiple segment detection results into a unified list. Use when you need to merge segments from different detectors, prepare removal lists for video processing, or consolidate detection outputs.","https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench\u002Fblob\u002Fmain\u002Ftasks\u002Fvideo-silence-remover\u002Fenvironment\u002Fskills\u002Fsegment-combiner\u002FSKILL.md",[],[121,154,97,155,156,157,158,159,21,160,161,117,120,162,163,28,164,165],"segment combiner","detection","prepare","merge","video processing","for","outputs","combine","processing","consolidate","results","list",{"skillNo":167,"name":168,"nameZh":6,"description":169,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":9,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":170,"categories":171,"keywords":175,"publishedAt":38},"01M6XEVMPVASGS2KX86QK4FMT7","report-generator","Generate compression reports for video processing. Use when you need to create structured JSON reports with duration statistics, compression ratios, and segment details after video processing.","https:\u002F\u002Fgithub.com\u002FWaaaalnut\u002Fskillsbench\u002Fblob\u002Fmain\u002Ftasks\u002Fvideo-silence-remover\u002Fenvironment\u002Fskills\u002Freport-generator\u002FSKILL.md",[172],{"categoryNo":173,"categoryName":174,"taxonomyType":79,"taxonomyTypeDesc":80},"01JBSEEDCAT000000000000034","行政与办公人员",[176,177,178,121,21,159,120,179,162,180,181,182,28,158,183,97,184,185,114],"generator","report generator","report","with","statistics","structured","reports","json","compression","generate"]