[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7CF448RCTDMTHR69E4GJXCP":3,"skill-same:01M7CF448RCTDMTHR69E4GJXCP":58},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":30,"commitSha":31,"skillDirectory":32,"forksCount":33,"repositoryHtmlUrl":34,"repositoryHomepageUrl":6,"ownerHtmlUrl":35,"ownerBlogUrl":6,"ownerDescription":36,"updatedAt":37,"install":38,"resources":51},"01M7CF448RCTDMTHR69E4GJXCP","zarr-python",null,"Chunked N-D arrays for cloud storage (Zarr-Python 3). Compressed arrays, parallel I\u002FO, S3\u002FGCS via fsspec, NumPy\u002FDask\u002FXarray compatible, for large-scale scientific computing pipelines.","om-scogo\u002Fskillsh-scraper","om-scogo","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160466840?v=4","01M79AWB6DFVCQ7RK70CS34B96","Om Achrekar","01M79AWB6ZM9SKR8KDH1WSSM22",0,"MIT","https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper\u002Fblob\u002Fmain\u002Fdata\u002Fskills\u002Fk-dense-ai-scientific-agent-skills\u002Fzarr-python\u002FSkill.md",[],[],"2026-11-15 14:19:17",false,"---\nname: zarr-python\ndescription: Chunked N-D arrays for cloud storage (Zarr-Python 3). Compressed arrays, parallel I\u002FO, S3\u002FGCS via fsspec, NumPy\u002FDask\u002FXarray compatible, for large-scale scientific computing pipelines.\nallowed-tools: Read Write Edit Bash\nlicense: MIT license\ncompatibility: Requires Python 3.12+ and zarr 3.x. Cloud I\u002FO needs zarr[remote] plus pinned s3fs or gcsfs. Legacy Zarr v2 workflows need exact 2.x pins on older Python.\nmetadata: {\"version\": \"1.1\", \"skill-author\": \"K-Dense Inc.\"}\n---\n\n# Zarr Python\n\n## Overview\n\nZarr is a Python library for storing large N-dimensional arrays with chunking and compression. Apply this skill for efficient parallel I\u002FO, cloud-native workflows, and seamless integration with NumPy, Dask, and Xarray.\n\n**Current upstream:** zarr **3.2.1** (released 2026-05-05). Docs: [zarr.readthedocs.io](https:\u002F\u002Fzarr.readthedocs.io\u002Fen\u002Fstable\u002F). New arrays default to **Zarr format 3**; set `zarr_format=2` for legacy interop. Zarr 3.2 adds rectilinear chunks and continues to refine the v3 codec pipeline. This skill is a **community guide** maintained by K-Dense Inc., not an official zarr-developers package.\n\n## Quick Start\n\n### Installation\n\n```bash\nuv pip install \"zarr==3.2.1\"\n```\n\nRequires **Python 3.12+** and NumPy 2.0+ for current stable Zarr-Python. For remote stores (S3, GCS, HTTP), pin the optional extras\u002Fbackends in your project lockfile:\n\n```bash\nuv pip install \"zarr[remote]==3.2.1\" \"s3fs==2026.4.0\" \"gcsfs==2026.5.0\"\n```\n\nUse a version range such as `zarr>=3,\u003C4` only when your project has a committed lockfile and compatibility tests. For Zarr-Python 2 \u002F Python 3.10–3.11 workflows, choose an exact `zarr==2.x.y` patch version from the support-v2 release notes and commit the resulting lockfile.\n\n### Basic Array Creation\n\n```python\nimport zarr\nimport numpy as np\n\n# Create a 2D array with chunking and compression\nz = zarr.create_array(\n    store=\"data\u002Fmy_array.zarr\",\n    shape=(10000, 10000),\n    chunks=(1000, 1000),\n    dtype=\"f4\"\n)\n\n# Write data using NumPy-style indexing\nz[:, :] = np.random.random((10000, 10000))\n\n# Read data\ndata = z[0:100, 0:100]  # Returns NumPy array\n```\n\n## Core Operations\n\n### Creating Arrays\n\nZarr provides multiple convenience functions for array creation:\n\n```python\n# Create empty array\nz = zarr.zeros(shape=(10000, 10000), chunks=(1000, 1000), dtype='f4',\n               store='data.zarr')\n\n# Create filled arrays\nz = zarr.ones((5000, 5000), chunks=(500, 500))\nz = zarr.full((1000, 1000), fill_value=42, chunks=(100, 100))\n\n# Create from existing data\ndata = np.arange(10000).reshape(100, 100)\nz = zarr.array(data, chunks=(10, 10), store='data.zarr')\n\n# Create like another array\nz2 = zarr.zeros_like(z)  # Matches shape, chunks, dtype of z\n```\n\n### Opening Existing Arrays\n\n```python\n# Open array (read\u002Fwrite mode by default)\nz = zarr.open_array('data.zarr', mode='r+')\n\n# Read-only mode\nz = zarr.open_array('data.zarr', mode='r')\n\n# The open() function auto-detects arrays vs groups\nz = zarr.open('data.zarr')  # Returns Array or Group\n```\n\n### Reading and Writing Data\n\nZarr arrays support NumPy-like indexing:\n\n```python\n# Write entire array\nz[:] = 42\n\n# Write slices\nz[0, :] = np.arange(100)\nz[10:20, 50:60] = np.random.random((10, 10))\n\n# Read data (returns NumPy array)\ndata = z[0:100, 0:100]\nrow = z[5, :]\n\n# Advanced indexing\nz.vindex[[0, 5, 10], [2, 8, 15]]  # Coordinate indexing\nz.oindex[0:10, [5, 10, 15]]       # Orthogonal indexing\nz.blocks[0, 0]                     # Block\u002Fchunk indexing\n```\n\n### Resizing and Appending\n\n```python\n# Resize array (v3: pass shape as a tuple)\nz.resize((15000, 15000))\n\n# Append data along an axis\nz.append(np.random.random((1000, 10000)), axis=0)  # Adds rows\n```\n\n## Chunking Strategies\n\nChunking is critical for performance. Choose chunk sizes and shapes based on access patterns.\n\n### Chunk Size Guidelines\n\n- **Minimum chunk size**: 1 MB recommended for optimal performance\n- **Balance**: Larger chunks = fewer metadata operations; smaller chunks = better parallel access\n- **Memory consideration**: Entire chunks must fit in memory during compression\n\n```python\n# Configure chunk size (aim for ~1MB per chunk)\n# For float32 data: 1MB = 262,144 elements = 512×512 array\nz = zarr.zeros(\n    shape=(10000, 10000),\n    chunks=(512, 512),  # ~1MB chunks\n    dtype='f4'\n)\n```\n\n### Aligning Chunks with Access Patterns\n\n**Critical**: Chunk shape dramatically affects performance based on how data is accessed.\n\n```python\n# If accessing rows frequently (first dimension)\nz = zarr.zeros((10000, 10000), chunks=(10, 10000))  # Chunk spans columns\n\n# If accessing columns frequently (second dimension)\nz = zarr.zeros((10000, 10000), chunks=(10000, 10))  # Chunk spans rows\n\n# For mixed access patterns (balanced approach)\nz = zarr.zeros((10000, 10000), chunks=(1000, 1000))  # Square chunks\n```\n\n**Performance example**: For a (200, 200, 200) array, reading along the first dimension:\n- Using chunks (1, 200, 200): ~107ms\n- Using chunks (200, 200, 1): ~1.65ms (65× faster!)\n\n### Rectilinear Chunks and Sharding\n\nZarr 3.2 supports **rectilinear chunks** for uneven grids. Pass nested chunk lengths when a dimension has variable tile sizes:\n\n```python\nz = zarr.create_array(\n    store=\"rectilinear.zarr\",\n    shape=(60, 100),\n    chunks=([10, 20, 30], [50, 50]),\n    dtype=\"f4\",\n)\n```\n\nWhen arrays have millions of small chunks, use **sharding** to group chunks into larger storage objects:\n\n```python\n# Create array with sharding\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(100000, 100000),\n    chunks=(100, 100),  # Small chunks for access\n    shards=(1000, 1000),  # Groups 100 chunks per shard\n    dtype='f4'\n)\n```\n\n**Benefits**:\n- Reduces file system overhead from millions of small files\n- Improves cloud storage performance (fewer object requests)\n- Prevents filesystem block size waste\n\n**Important**: Entire shards must fit in memory before writing.\n\n## Compression\n\nZarr applies compression per chunk to reduce storage while maintaining fast access.\n\n### Configuring Compression\n\n```python\nfrom zarr.codecs import BloscCodec, BloscShuffle, GzipCodec\n\n# Default: Blosc with Zstandard\nz = zarr.zeros((1000, 1000), chunks=(100, 100))  # Uses default compression\n\n# Configure Blosc compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    compressors=BloscCodec(cname='zstd', clevel=5, shuffle=BloscShuffle.bitshuffle)\n)\n\n# Available Blosc compressors: 'blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd'\n\n# Use Gzip compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    compressors=GzipCodec(level=6)\n)\n\n# Disable compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    compressors=None\n)\n```\n\n### Compression Performance Tips\n\n- **Blosc** (default): Fast compression\u002Fdecompression, good for interactive workloads\n- **Zstandard**: Better compression ratios, slightly slower than LZ4\n- **Gzip**: Maximum compression, slower performance\n- **LZ4**: Fastest compression, lower ratios\n- **Shuffle**: Enable shuffle filter for better compression on numeric data\n\n```python\n# Optimal for numeric scientific data\ncompressors=BloscCodec(cname='zstd', clevel=5, shuffle=BloscShuffle.bitshuffle)\n\n# Optimal for speed\ncompressors=BloscCodec(cname='lz4', clevel=1)\n\n# Optimal for compression ratio\ncompressors=GzipCodec(level=9)\n```\n\n## Storage Backends\n\nZarr supports multiple storage backends through a flexible storage interface.\n\n### Local Filesystem (Default)\n\n```python\nfrom zarr.storage import LocalStore\n\n# Explicit store creation\nstore = LocalStore('data\u002Fmy_array.zarr')\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n\n# Or use string path (creates LocalStore automatically)\nz = zarr.open_array('data\u002Fmy_array.zarr', mode='w', shape=(1000, 1000),\n                    chunks=(100, 100))\n```\n\n### In-Memory Storage\n\n```python\nfrom zarr.storage import MemoryStore\n\n# Create in-memory store\nstore = MemoryStore()\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n\n# Data exists only in memory, not persisted\n```\n\n### ZIP File Storage\n\n```python\nfrom zarr.storage import ZipStore\n\n# Write to ZIP file\nstore = ZipStore('data.zip', mode='w')\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\nz[:] = np.random.random((1000, 1000))\nstore.close()  # IMPORTANT: Must close ZipStore\n\n# Read from ZIP file\nstore = ZipStore('data.zip', mode='r')\nz = zarr.open_array(store=store)\ndata = z[:]\nstore.close()\n```\n\n### Cloud Storage (S3, GCS)\n\nZarr 3 uses **fsspec** backends via URI strings or `FsspecStore` (preferred over legacy `S3Map`\u002F`GCSMap`).\n\n```python\nimport zarr\n\n# S3 — prefer IAM roles\u002Fprofiles; fsspec handles provider credential discovery.\n# Never print, log, or copy credential values into prompts or notebooks.\nz = zarr.create_array(\n    store=\"s3:\u002F\u002Fmy-bucket\u002Fpath\u002Fto\u002Farray.zarr\",\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype=\"f4\",\n    storage_options={\"anon\": False},\n)\nz[:] = data\n\n# GCS — prefer workload identity or gcloud application-default credentials.\nz = zarr.open_array(\n    \"gs:\u002F\u002Fmy-bucket\u002Fpath\u002Fto\u002Farray.zarr\",\n    mode=\"r\",\n    storage_options={\"project\": \"my-project\"},\n)\n\n# Explicit store (any fsspec filesystem)\nfrom zarr.storage import FsspecStore\nstore = FsspecStore.from_url(\"s3:\u002F\u002Fmy-bucket\u002Fdata.zarr\", storage_options={\"anon\": False})\nroot = zarr.open_group(store=store, mode=\"r+\")\n```\n\nCloud backends read credentials through the provider SDK\u002Ffsspec backend. Do not inspect broad `.env` files; if a user explicitly needs help debugging auth, ask for redacted configuration and read only the named provider variables they approve. Treat all `import zarr`, `import dask`, `import h5py`, and `import xarray` examples as third-party package imports, not bundled script files.\n\n**Cloud Storage Best Practices**:\n- Use consolidated metadata to reduce latency: `zarr.consolidate_metadata(store)`\n- Align chunk sizes with cloud object sizing (typically 5-100 MB optimal)\n- Enable parallel writes using Dask for large-scale data\n- Consider sharding to reduce number of objects\n\n## Groups and Hierarchies\n\nGroups organize multiple arrays hierarchically, similar to directories or HDF5 groups.\n\n### Creating and Using Groups\n\n```python\n# Create root group\nroot = zarr.group(store='data\u002Fhierarchy.zarr')\n\n# Create sub-groups\ntemperature = root.create_group('temperature')\nprecipitation = root.create_group('precipitation')\n\n# Create arrays within groups\ntemp_array = temperature.create_array(\n    name='t2m',\n    shape=(365, 720, 1440),\n    chunks=(1, 720, 1440),\n    dtype='f4'\n)\n\nprecip_array = precipitation.create_array(\n    name='prcp',\n    shape=(365, 720, 1440),\n    chunks=(1, 720, 1440),\n    dtype='f4'\n)\n\n# Access using paths\narray = root['temperature\u002Ft2m']\n\n# Visualize hierarchy\nprint(root.tree())\n# Output:\n# \u002F\n#  ├── temperature\n#  │   └── t2m (365, 720, 1440) f4\n#  └── precipitation\n#      └── prcp (365, 720, 1440) f4\n```\n\n### Group API (v3)\n\nUse `create_array` \u002F `require_array` (h5py-style `create_dataset` \u002F `require_dataset` were removed in v3):\n\n```python\nroot = zarr.group('data.zarr')\narr = root.create_array('my_data', shape=(1000, 1000), chunks=(100, 100), dtype='f4')\n\ngrp = root.require_group('subgroup')\narr2 = grp.require_array('array', shape=(500, 500), chunks=(50, 50), dtype='i4')\n```\n\n## Attributes and Metadata\n\nAttach custom metadata to arrays and groups using attributes:\n\n```python\n# Add attributes to array\nz = zarr.zeros((1000, 1000), chunks=(100, 100))\nz.attrs['description'] = 'Temperature data in Kelvin'\nz.attrs['units'] = 'K'\nz.attrs['created'] = '2024-01-15'\nz.attrs['processing_version'] = 2.1\n\n# Attributes are stored as JSON\nprint(z.attrs['units'])  # Output: K\n\n# Add attributes to groups\nroot = zarr.group('data.zarr')\nroot.attrs['project'] = 'Climate Analysis'\nroot.attrs['institution'] = 'Research Institute'\n\n# Attributes persist with the array\u002Fgroup\nz2 = zarr.open('data.zarr')\nprint(z2.attrs['description'])\n```\n\n**Important**: Attributes must be JSON-serializable (strings, numbers, lists, dicts, booleans, null).\n\n## Integration with NumPy, Dask, and Xarray\n\n### NumPy Integration\n\nZarr arrays implement the NumPy array interface:\n\n```python\nimport numpy as np\nimport zarr\n\nz = zarr.zeros((1000, 1000), chunks=(100, 100))\n\n# Use NumPy functions directly\nresult = np.sum(z, axis=0)  # NumPy operates on Zarr array\nmean = np.mean(z[:100, :100])\n\n# Convert to NumPy array\nnumpy_array = z[:]  # Loads entire array into memory\n```\n\n### Dask Integration\n\nDask provides lazy, parallel computation on Zarr arrays:\n\n```python\nimport dask.array as da\nimport zarr\n\n# Create large Zarr array\nz = zarr.open('data.zarr', mode='w', shape=(100000, 100000),\n              chunks=(1000, 1000), dtype='f4')\n\n# Load as Dask array (lazy, no data loaded)\ndask_array = da.from_zarr('data.zarr')\n\n# Perform computations (parallel, out-of-core)\nresult = dask_array.mean(axis=0).compute()  # Parallel computation\n\n# Write Dask array to Zarr\nlarge_array = da.random.random((100000, 100000), chunks=(1000, 1000))\nda.to_zarr(large_array, 'output.zarr')\n```\n\n**Benefits**:\n- Process datasets larger than memory\n- Automatic parallel computation across chunks\n- Efficient I\u002FO with chunked storage\n\n### Xarray Integration\n\nXarray provides labeled, multidimensional arrays with Zarr backend:\n\n```python\nimport xarray as xr\nimport zarr\n\n# Open Zarr store as Xarray Dataset (lazy loading)\nds = xr.open_zarr('data.zarr')\n\n# Dataset includes coordinates and metadata\nprint(ds)\n\n# Access variables\ntemperature = ds['temperature']\n\n# Perform labeled operations\nsubset = ds.sel(time='2024-01', lat=slice(30, 60))\n\n# Write Xarray Dataset to Zarr\nds.to_zarr('output.zarr')\n\n# Create from scratch with coordinates\nds = xr.Dataset(\n    {\n        'temperature': (['time', 'lat', 'lon'], data),\n        'precipitation': (['time', 'lat', 'lon'], data2)\n    },\n    coords={\n        'time': pd.date_range('2024-01-01', periods=365),\n        'lat': np.arange(-90, 91, 1),\n        'lon': np.arange(-180, 180, 1)\n    }\n)\nds.to_zarr('climate_data.zarr')\n```\n\n**Benefits**:\n- Named dimensions and coordinates\n- Label-based indexing and selection\n- Integration with pandas for time series\n- NetCDF-like interface familiar to climate\u002Fgeospatial scientists\n\n## Parallel Computing and Thread Safety\n\nZarr uses async I\u002FO internally. Tune concurrency for remote storage or Dask-heavy workloads:\n\n```python\nimport zarr\n\n# Higher values can improve remote throughput; lower values reduce pressure\n# when Dask already supplies many worker threads.\nzarr.config.set({\n    \"async.concurrency\": 8,\n    \"threading.max_workers\": 8,\n})\n```\n\nThe old `synchronizer` argument (`ThreadSynchronizer`, `ProcessSynchronizer`) is **not available in Zarr-Python 3**. Use these patterns instead:\n\n- **Reads:** always safe across threads\u002Fprocesses.\n- **Writes:** safe when each worker writes to **non-overlapping chunks**; most stores support atomic chunk writes.\n- **Overlapping writes:** coordinate externally (file locks, workflow design) until synchronizers return.\n\nFor Dask-heavy workloads, estimate total concurrent I\u002FO as roughly `dask_threads × async.concurrency` and lower Zarr's concurrency settings if the store or memory becomes saturated.\n\n## Consolidated Metadata\n\nFor hierarchical stores with many arrays, consolidate metadata into a single file to reduce I\u002FO operations:\n\n```python\nimport zarr\n\n# After creating arrays\u002Fgroups\nroot = zarr.group('data.zarr')\n# ... create multiple arrays\u002Fgroups ...\n\n# Consolidate metadata\nzarr.consolidate_metadata('data.zarr')\n\n# Open with consolidated metadata (faster, especially on cloud storage)\nroot = zarr.open_consolidated('data.zarr')\n```\n\n**Benefits**:\n- Reduces metadata read operations from N (one per array) to 1\n- Critical for cloud storage (reduces latency)\n- Speeds up `tree()` operations and group traversal\n\n**Cautions**:\n- Metadata can become stale if arrays update without re-consolidation\n- Not suitable for frequently-updated datasets\n- Multi-writer scenarios may have inconsistent reads\n\n## Performance Optimization\n\n### Checklist for Optimal Performance\n\n1. **Chunk Size**: Aim for 1-10 MB per chunk\n   ```python\n   # For float32: 1MB = 262,144 elements\n   chunks = (512, 512)  # 512×512×4 bytes = ~1MB\n   ```\n\n2. **Chunk Shape**: Align with access patterns\n   ```python\n   # Row-wise access → chunk spans columns: (small, large)\n   # Column-wise access → chunk spans rows: (large, small)\n   # Random access → balanced: (medium, medium)\n   ```\n\n3. **Compression**: Choose based on workload\n   ```python\n   # Interactive\u002Ffast: BloscCodec(cname='lz4')\n   # Balanced: BloscCodec(cname='zstd', clevel=5)\n   # Maximum compression: GzipCodec(level=9)\n   ```\n\n4. **Storage Backend**: Match to environment\n   ```python\n   # Local: LocalStore (default)\n   # Cloud: fsspec URIs or FsspecStore + consolidated metadata\n   # Temporary: MemoryStore\n   ```\n\n5. **Sharding**: Use for large-scale datasets\n   ```python\n   # When you have millions of small chunks\n   shards=(10*chunk_size, 10*chunk_size)\n   ```\n\n6. **Parallel I\u002FO**: Use Dask for large operations\n   ```python\n   import dask.array as da\n   dask_array = da.from_zarr('data.zarr')\n   result = dask_array.compute(scheduler='threads', num_workers=8)\n   ```\n\n### Profiling and Debugging\n\n```python\n# Print detailed array information\nprint(z.info)\n\n# Output includes:\n# - Type, shape, chunks, dtype\n# - Serializer and compressors\n# - Storage size (compressed vs uncompressed)\n# - Storage location\n\n# Check storage size\nprint(f\"Compressed size: {z.nbytes_stored \u002F 1e6:.2f} MB\")\nprint(f\"Uncompressed size: {z.nbytes \u002F 1e6:.2f} MB\")\nprint(f\"Compression ratio: {z.nbytes \u002F z.nbytes_stored:.2f}x\")\n```\n\n## Common Patterns and Best Practices\n\n### Pattern: Time Series Data\n\n```python\n# Store time series with time as first dimension\n# This allows efficient appending of new time steps\nz = zarr.open('timeseries.zarr', mode='a',\n              shape=(0, 720, 1440),  # Start with 0 time steps\n              chunks=(1, 720, 1440),  # One time step per chunk\n              dtype='f4')\n\n# Append new time steps\nnew_data = np.random.random((1, 720, 1440))\nz.append(new_data, axis=0)\n```\n\n### Pattern: Large Matrix Operations\n\n```python\nimport dask.array as da\n\n# Create large matrix in Zarr\nz = zarr.open('matrix.zarr', mode='w',\n              shape=(100000, 100000),\n              chunks=(1000, 1000),\n              dtype='f8')\n\n# Use Dask for parallel computation\ndask_z = da.from_zarr('matrix.zarr')\nresult = (dask_z @ dask_z.T).compute()  # Parallel matrix multiply\n```\n\n### Pattern: Cloud-Native Workflow\n\n```python\nimport zarr\n\npath = \"s3:\u002F\u002Fmy-bucket\u002Fdata.zarr\"\nz = zarr.create_array(\n    store=path,\n    shape=(10000, 10000),\n    chunks=(500, 500),\n    dtype=\"f4\",\n    storage_options={\"anon\": False},\n)\nz[:] = data\n\nzarr.consolidate_metadata(path)\nz_read = zarr.open_consolidated(path, storage_options={\"anon\": False})\nsubset = z_read[0:100, 0:100]\n```\n\n### Pattern: Format Conversion\n\n```python\n# HDF5 to Zarr\nimport h5py\nimport zarr\n\nwith h5py.File('data.h5', 'r') as h5:\n    dataset = h5['dataset_name']\n    z = zarr.array(dataset[:],\n                   chunks=(1000, 1000),\n                   store='data.zarr')\n\n# NumPy to Zarr\nimport numpy as np\ndata = np.load('data.npy')\nz = zarr.array(data, chunks='auto', store='data.zarr')\n\n# Zarr to NetCDF (via Xarray)\nimport xarray as xr\nds = xr.open_zarr('data.zarr')\nds.to_netcdf('data.nc')\n```\n\n## Common Issues and Solutions\n\n### Issue: Slow Performance\n\n**Diagnosis**: Check chunk size and alignment\n```python\nprint(z.chunks)  # Are chunks appropriate size?\nprint(z.info)    # Check compression ratio\n```\n\n**Solutions**:\n- Increase chunk size to 1-10 MB\n- Align chunks with access pattern\n- Try different compression codecs\n- Use Dask for parallel operations\n\n### Issue: High Memory Usage\n\n**Cause**: Loading entire array or large chunks into memory\n\n**Solutions**:\n```python\n# Don't load entire array\n# Bad: data = z[:]\n# Good: Process in chunks\nfor i in range(0, z.shape[0], 1000):\n    chunk = z[i:i+1000, :]\n    process(chunk)\n\n# Or use Dask for automatic chunking\nimport dask.array as da\ndask_z = da.from_zarr('data.zarr')\nresult = dask_z.mean().compute()  # Processes in chunks\n```\n\n### Issue: Cloud Storage Latency\n\n**Solutions**:\n```python\n# 1. Consolidate metadata\nzarr.consolidate_metadata(store)\nz = zarr.open_consolidated(store)\n\n# 2. Use appropriate chunk sizes (5-100 MB for cloud)\nchunks = (2000, 2000)  # Larger chunks for cloud\n\n# 3. Enable sharding\nshards = (10000, 10000)  # Groups many chunks\n```\n\n### Issue: Concurrent Write Conflicts\n\n**Solution**: Design workflows so each process\u002Fthread writes to separate chunks. Zarr-Python 3 does not yet support `ThreadSynchronizer` \u002F `ProcessSynchronizer`; see `references\u002Fv3_migration.md`.\n\n## Additional Resources\n\n### Bundled references\n\n| File | Contents |\n|------|----------|\n| `references\u002Fapi_reference.md` | Function signatures, stores, codecs, indexing |\n| `references\u002Fv3_migration.md` | Zarr-Python 2→3 breaking changes and WIP features |\n\n### Official upstream\n\n- **Documentation**: https:\u002F\u002Fzarr.readthedocs.io\u002Fen\u002Fstable\u002F\n- **3.0 migration guide**: https:\u002F\u002Fzarr.readthedocs.io\u002Fen\u002Fstable\u002Fuser-guide\u002Fv3_migration\u002F\n- **Storage backends**: https:\u002F\u002Fzarr.readthedocs.io\u002Fen\u002Fstable\u002Fuser-guide\u002Fstorage\u002F\n- **Zarr specifications**: https:\u002F\u002Fzarr-specs.readthedocs.io\u002F\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fzarr-developers\u002Fzarr-python\n- **Developer chat**: https:\u002F\u002Fossci.zulipchat.com\u002F#narrow\u002Fchannel\u002F423692-Zarr-Python\n\n**Related libraries:** [Xarray](https:\u002F\u002Fdocs.xarray.dev\u002F), [Dask](https:\u002F\u002Fdocs.dask.org\u002F), [NumCodecs](https:\u002F\u002Fnumcodecs.readthedocs.io\u002F)\n\n","# Zarr Python\n\n## Overview\n\nZarr is a Python library for storing large N-dimensional arrays with chunking and compression. Apply this skill for efficient parallel I\u002FO, cloud-native workflows, and seamless integration with NumPy, Dask, and Xarray.\n\n**Current upstream:** zarr **3.2.1** (released 2026-05-05). Docs: [zarr.readthedocs.io](https:\u002F\u002Fzarr.readthedocs.io\u002Fen\u002Fstable\u002F). New arrays default to **Zarr format 3**; set `zarr_format=2` for legacy interop. Zarr 3.2 adds rectilinear chunks and continues to refine the v3 codec pipeline. This skill is a **community guide** maintained by K-Dense Inc., not an official zarr-developers package.\n\n## Quick Start\n\n### Installation\n\n```bash\nuv pip install \"zarr==3.2.1\"\n```\n\nRequires **Python 3.12+** and NumPy 2.0+ for current stable Zarr-Python. For remote stores (S3, GCS, HTTP), pin the optional extras\u002Fbackends in your project lockfile:\n\n```bash\nuv pip install \"zarr[remote]==3.2.1\" \"s3fs==2026.4.0\" \"gcsfs==2026.5.0\"\n```\n\nUse a version range such as `zarr>=3,\u003C4` only when your project has a committed lockfile and compatibility tests. For Zarr-Python 2 \u002F Python 3.10–3.11 workflows, choose an exact `zarr==2.x.y` patch version from the support-v2 release notes and commit the resulting lockfile.\n\n### Basic Array Creation\n\n```python\nimport zarr\nimport numpy as np\n\n# Create a 2D array with chunking and compression\nz = zarr.create_array(\n    store=\"data\u002Fmy_array.zarr\",\n    shape=(10000, 10000),\n    chunks=(1000, 1000),\n    dtype=\"f4\"\n)\n\n# Write data using NumPy-style indexing\nz[:, :] = np.random.random((10000, 10000))\n\n# Read data\ndata = z[0:100, 0:100]  # Returns NumPy array\n```\n\n## Core Operations\n\n### Creating Arrays\n\nZarr provides multiple convenience functions for array creation:\n\n```python\n# Create empty array\nz = zarr.zeros(shape=(10000, 10000), chunks=(1000, 1000), dtype='f4',\n               store='data.zarr')\n\n# Create filled arrays\nz = zarr.ones((5000, 5000), chunks=(500, 500))\nz = zarr.full((1000, 1000), fill_value=42, chunks=(100, 100))\n\n# Create from existing data\ndata = np.arange(10000).reshape(100, 100)\nz = zarr.array(data, chunks=(10, 10), store='data.zarr')\n\n# Create like another array\nz2 = zarr.zeros_like(z)  # Matches shape, chunks, dtype of z\n```\n\n### Opening Existing Arrays\n\n```python\n# Open array (read\u002Fwrite mode by default)\nz = zarr.open_array('data.zarr', mode='r+')\n\n# Read-only mode\nz = zarr.open_array('data.zarr', mode='r')\n\n# The open() function auto-detects arrays vs groups\nz = zarr.open('data.zarr')  # Returns Array or Group\n```\n\n### Reading and Writing Data\n\nZarr arrays support NumPy-like indexing:\n\n```python\n# Write entire array\nz[:] = 42\n\n# Write slices\nz[0, :] = np.arange(100)\nz[10:20, 50:60] = np.random.random((10, 10))\n\n# Read data (returns NumPy array)\ndata = z[0:100, 0:100]\nrow = z[5, :]\n\n# Advanced indexing\nz.vindex[[0, 5, 10], [2, 8, 15]]  # Coordinate indexing\nz.oindex[0:10, [5, 10, 15]]       # Orthogonal indexing\nz.blocks[0, 0]                     # Block\u002Fchunk indexing\n```\n\n### Resizing and Appending\n\n```python\n# Resize array (v3: pass shape as a tuple)\nz.resize((15000, 15000))\n\n# Append data along an axis\nz.append(np.random.random((1000, 10000)), axis=0)  # Adds rows\n```\n\n## Chunking Strategies\n\nChunking is critical for performance. Choose chunk sizes and shapes based on access patterns.\n\n### Chunk Size Guidelines\n\n- **Minimum chunk size**: 1 MB recommended for optimal performance\n- **Balance**: Larger chunks = fewer metadata operations; smaller chunks = better parallel access\n- **Memory consideration**: Entire chunks must fit in memory during compression\n\n```python\n# Configure chunk size (aim for ~1MB per chunk)\n# For float32 data: 1MB = 262,144 elements = 512×512 array\nz = zarr.zeros(\n    shape=(10000, 10000),\n    chunks=(512, 512),  # ~1MB chunks\n    dtype='f4'\n)\n```\n\n### Aligning Chunks with Access Patterns\n\n**Critical**: Chunk shape dramatically affects performance based on how data is accessed.\n\n```python\n# If accessing rows frequently (first dimension)\nz = zarr.zeros((10000, 10000), chunks=(10, 10000))  # Chunk spans columns\n\n# If accessing columns frequently (second dimension)\nz = zarr.zeros((10000, 10000), chunks=(10000, 10))  # Chunk spans rows\n\n# For mixed access patterns (balanced approach)\nz = zarr.zeros((10000, 10000), chunks=(1000, 1000))  # Square chunks\n```\n\n**Performance example**: For a (200, 200, 200) array, reading along the first dimension:\n- Using chunks (1, 200, 200): ~107ms\n- Using chunks (200, 200, 1): ~1.65ms (65× faster!)\n\n### Rectilinear Chunks and Sharding\n\nZarr 3.2 supports **rectilinear chunks** for uneven grids. Pass nested chunk lengths when a dimension has variable tile sizes:\n\n```python\nz = zarr.create_array(\n    store=\"rectilinear.zarr\",\n    shape=(60, 100),\n    chunks=([10, 20, 30], [50, 50]),\n    dtype=\"f4\",\n)\n```\n\nWhen arrays have millions of small chunks, use **sharding** to group chunks into larger storage objects:\n\n```python\n# Create array with sharding\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(100000, 100000),\n    chunks=(100, 100),  # Small chunks for access\n    shards=(1000, 1000),  # Groups 100 chunks per shard\n    dtype='f4'\n)\n```\n\n**Benefits**:\n- Reduces file system overhead from millions of small files\n- Improves cloud storage performance (fewer object requests)\n- Prevents filesystem block size waste\n\n**Important**: Entire shards must fit in memory before writing.\n\n## Compression\n\nZarr applies compression per chunk to reduce storage while maintaining fast access.\n\n### Configuring Compression\n\n```python\nfrom zarr.codecs import BloscCodec, BloscShuffle, GzipCodec\n\n# Default: Blosc with Zstandard\nz = zarr.zeros((1000, 1000), chunks=(100, 100))  # Uses default compression\n\n# Configure Blosc compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    compressors=BloscCodec(cname='zstd', clevel=5, shuffle=BloscShuffle.bitshuffle)\n)\n\n# Available Blosc compressors: 'blosclz', 'lz4', 'lz4hc', 'snappy', 'zlib', 'zstd'\n\n# Use Gzip compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    compressors=GzipCodec(level=6)\n)\n\n# Disable compression\nz = zarr.create_array(\n    store='data.zarr',\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype='f4',\n    compressors=None\n)\n```\n\n### Compression Performance Tips\n\n- **Blosc** (default): Fast compression\u002Fdecompression, good for interactive workloads\n- **Zstandard**: Better compression ratios, slightly slower than LZ4\n- **Gzip**: Maximum compression, slower performance\n- **LZ4**: Fastest compression, lower ratios\n- **Shuffle**: Enable shuffle filter for better compression on numeric data\n\n```python\n# Optimal for numeric scientific data\ncompressors=BloscCodec(cname='zstd', clevel=5, shuffle=BloscShuffle.bitshuffle)\n\n# Optimal for speed\ncompressors=BloscCodec(cname='lz4', clevel=1)\n\n# Optimal for compression ratio\ncompressors=GzipCodec(level=9)\n```\n\n## Storage Backends\n\nZarr supports multiple storage backends through a flexible storage interface.\n\n### Local Filesystem (Default)\n\n```python\nfrom zarr.storage import LocalStore\n\n# Explicit store creation\nstore = LocalStore('data\u002Fmy_array.zarr')\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n\n# Or use string path (creates LocalStore automatically)\nz = zarr.open_array('data\u002Fmy_array.zarr', mode='w', shape=(1000, 1000),\n                    chunks=(100, 100))\n```\n\n### In-Memory Storage\n\n```python\nfrom zarr.storage import MemoryStore\n\n# Create in-memory store\nstore = MemoryStore()\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\n\n# Data exists only in memory, not persisted\n```\n\n### ZIP File Storage\n\n```python\nfrom zarr.storage import ZipStore\n\n# Write to ZIP file\nstore = ZipStore('data.zip', mode='w')\nz = zarr.open_array(store=store, mode='w', shape=(1000, 1000), chunks=(100, 100))\nz[:] = np.random.random((1000, 1000))\nstore.close()  # IMPORTANT: Must close ZipStore\n\n# Read from ZIP file\nstore = ZipStore('data.zip', mode='r')\nz = zarr.open_array(store=store)\ndata = z[:]\nstore.close()\n```\n\n### Cloud Storage (S3, GCS)\n\nZarr 3 uses **fsspec** backends via URI strings or `FsspecStore` (preferred over legacy `S3Map`\u002F`GCSMap`).\n\n```python\nimport zarr\n\n# S3 — prefer IAM roles\u002Fprofiles; fsspec handles provider credential discovery.\n# Never print, log, or copy credential values into prompts or notebooks.\nz = zarr.create_array(\n    store=\"s3:\u002F\u002Fmy-bucket\u002Fpath\u002Fto\u002Farray.zarr\",\n    shape=(1000, 1000),\n    chunks=(100, 100),\n    dtype=\"f4\",\n    storage_options={\"anon\": False},\n)\nz[:] = data\n\n# GCS — prefer workload identity or gcloud application-default credentials.\nz = zarr.open_array(\n    \"gs:\u002F\u002Fmy-bucket\u002Fpath\u002Fto\u002Farray.zarr\",\n    mode=\"r\",\n    storage_options={\"project\": \"my-project\"},\n)\n\n# Explicit store (any fsspec filesystem)\nfrom zarr.storage import FsspecStore\nstore = FsspecStore.from_url(\"s3:\u002F\u002Fmy-bucket\u002Fdata.zarr\", storage_options={\"anon\": False})\nroot = zarr.open_group(store=store, mode=\"r+\")\n```\n\nCloud backends read credentials through the provider SDK\u002Ffsspec backend. Do not inspect broad `.env` files; if a user explicitly needs help debugging auth, ask for redacted configuration and read only the named provider variables they approve. Treat all `import zarr`, `import dask`, `import h5py`, and `import xarray` examples as third-party package imports, not bundled script files.\n\n**Cloud Storage Best Practices**:\n- Use consolidated metadata to reduce latency: `zarr.consolidate_metadata(store)`\n- Align chunk sizes with cloud object sizing (typically 5-100 MB optimal)\n- Enable parallel writes using Dask for large-scale data\n- Consider sharding to reduce number of objects\n\n## Groups and Hierarchies\n\nGroups organize multiple arrays hierarchically, similar to directories or HDF5 groups.\n\n### Creating and Using Groups\n\n```python\n# Create root group\nroot = zarr.group(store='data\u002Fhierarchy.zarr')\n\n# Create sub-groups\ntemperature = root.create_group('temperature')\nprecipitation = root.create_group('precipitation')\n\n# Create arrays within groups\ntemp_array = temperature.create_array(\n    name='t2m',\n    shape=(365, 720, 1440),\n    chunks=(1, 720, 1440),\n    dtype='f4'\n)\n\nprecip_array = precipitation.create_array(\n    name='prcp',\n    shape=(365, 720, 1440),\n    chunks=(1, 720, 1440),\n    dtype='f4'\n)\n\n# Access using paths\narray = root['temperature\u002Ft2m']\n\n# Visualize hierarchy\nprint(root.tree())\n# Output:\n# \u002F\n#  ├── temperature\n#  │   └── t2m (365, 720, 1440) f4\n#  └── precipitation\n#      └── prcp (365, 720, 1440) f4\n```\n\n### Group API (v3)\n\nUse `create_array` \u002F `require_array` (h5py-style `create_dataset` \u002F `require_dataset` were removed in v3):\n\n```python\nroot = zarr.group('data.zarr')\narr = root.create_array('my_data', shape=(1000, 1000), chunks=(100, 100), dtype='f4')\n\ngrp = root.require_group('subgroup')\narr2 = grp.require_array('array', shape=(500, 500), chunks=(50, 50), dtype='i4')\n```\n\n## Attributes and Metadata\n\nAttach custom metadata to arrays and groups using attributes:\n\n```python\n# Add attributes to array\nz = zarr.zeros((1000, 1000), chunks=(100, 100))\nz.attrs['description'] = 'Temperature data in Kelvin'\nz.attrs['units'] = 'K'\nz.attrs['created'] = '2024-01-15'\nz.attrs['processing_version'] = 2.1\n\n# Attributes are stored as JSON\nprint(z.attrs['units'])  # Output: K\n\n# Add attributes to groups\nroot = zarr.group('data.zarr')\nroot.attrs['project'] = 'Climate Analysis'\nroot.attrs['institution'] = 'Research Institute'\n\n# Attributes persist with the array\u002Fgroup\nz2 = zarr.open('data.zarr')\nprint(z2.attrs['description'])\n```\n\n**Important**: Attributes must be JSON-serializable (strings, numbers, lists, dicts, booleans, null).\n\n## Integration with NumPy, Dask, and Xarray\n\n### NumPy Integration\n\nZarr arrays implement the NumPy array interface:\n\n```python\nimport numpy as np\nimport zarr\n\nz = zarr.zeros((1000, 1000), chunks=(100, 100))\n\n# Use NumPy functions directly\nresult = np.sum(z, axis=0)  # NumPy operates on Zarr array\nmean = np.mean(z[:100, :100])\n\n# Convert to NumPy array\nnumpy_array = z[:]  # Loads entire array into memory\n```\n\n### Dask Integration\n\nDask provides lazy, parallel computation on Zarr arrays:\n\n```python\nimport dask.array as da\nimport zarr\n\n# Create large Zarr array\nz = zarr.open('data.zarr', mode='w', shape=(100000, 100000),\n              chunks=(1000, 1000), dtype='f4')\n\n# Load as Dask array (lazy, no data loaded)\ndask_array = da.from_zarr('data.zarr')\n\n# Perform computations (parallel, out-of-core)\nresult = dask_array.mean(axis=0).compute()  # Parallel computation\n\n# Write Dask array to Zarr\nlarge_array = da.random.random((100000, 100000), chunks=(1000, 1000))\nda.to_zarr(large_array, 'output.zarr')\n```\n\n**Benefits**:\n- Process datasets larger than memory\n- Automatic parallel computation across chunks\n- Efficient I\u002FO with chunked storage\n\n### Xarray Integration\n\nXarray provides labeled, multidimensional arrays with Zarr backend:\n\n```python\nimport xarray as xr\nimport zarr\n\n# Open Zarr store as Xarray Dataset (lazy loading)\nds = xr.open_zarr('data.zarr')\n\n# Dataset includes coordinates and metadata\nprint(ds)\n\n# Access variables\ntemperature = ds['temperature']\n\n# Perform labeled operations\nsubset = ds.sel(time='2024-01', lat=slice(30, 60))\n\n# Write Xarray Dataset to Zarr\nds.to_zarr('output.zarr')\n\n# Create from scratch with coordinates\nds = xr.Dataset(\n    {\n        'temperature': (['time', 'lat', 'lon'], data),\n        'precipitation': (['time', 'lat', 'lon'], data2)\n    },\n    coords={\n        'time': pd.date_range('2024-01-01', periods=365),\n        'lat': np.arange(-90, 91, 1),\n        'lon': np.arange(-180, 180, 1)\n    }\n)\nds.to_zarr('climate_data.zarr')\n```\n\n**Benefits**:\n- Named dimensions and coordinates\n- Label-based indexing and selection\n- Integration with pandas for time series\n- NetCDF-like interface familiar to climate\u002Fgeospatial scientists\n\n## Parallel Computing and Thread Safety\n\nZarr uses async I\u002FO internally. Tune concurrency for remote storage or Dask-heavy workloads:\n\n```python\nimport zarr\n\n# Higher values can improve remote throughput; lower values reduce pressure\n# when Dask already supplies many worker threads.\nzarr.config.set({\n    \"async.concurrency\": 8,\n    \"threading.max_workers\": 8,\n})\n```\n\nThe old `synchronizer` argument (`ThreadSynchronizer`, `ProcessSynchronizer`) is **not available in Zarr-Python 3**. Use these patterns instead:\n\n- **Reads:** always safe across threads\u002Fprocesses.\n- **Writes:** safe when each worker writes to **non-overlapping chunks**; most stores support atomic chunk writes.\n- **Overlapping writes:** coordinate externally (file locks, workflow design) until synchronizers return.\n\nFor Dask-heavy workloads, estimate total concurrent I\u002FO as roughly `dask_threads × async.concurrency` and lower Zarr's concurrency settings if the store or memory becomes saturated.\n\n## Consolidated Metadata\n\nFor hierarchical stores with many arrays, consolidate metadata into a single file to reduce I\u002FO operations:\n\n```python\nimport zarr\n\n# After creating arrays\u002Fgroups\nroot = zarr.group('data.zarr')\n# ... create multiple arrays\u002Fgroups ...\n\n# Consolidate metadata\nzarr.consolidate_metadata('data.zarr')\n\n# Open with consolidated metadata (faster, especially on cloud storage)\nroot = zarr.open_consolidated('data.zarr')\n```\n\n**Benefits**:\n- Reduces metadata read operations from N (one per array) to 1\n- Critical for cloud storage (reduces latency)\n- Speeds up `tree()` operations and group traversal\n\n**Cautions**:\n- Metadata can become stale if arrays update without re-consolidation\n- Not suitable for frequently-updated datasets\n- Multi-writer scenarios may have inconsistent reads\n\n## Performance Optimization\n\n### Checklist for Optimal Performance\n\n1. **Chunk Size**: Aim for 1-10 MB per chunk\n   ```python\n   # For float32: 1MB = 262,144 elements\n   chunks = (512, 512)  # 512×512×4 bytes = ~1MB\n   ```\n\n2. **Chunk Shape**: Align with access patterns\n   ```python\n   # Row-wise access → chunk spans columns: (small, large)\n   # Column-wise access → chunk spans rows: (large, small)\n   # Random access → balanced: (medium, medium)\n   ```\n\n3. **Compression**: Choose based on workload\n   ```python\n   # Interactive\u002Ffast: BloscCodec(cname='lz4')\n   # Balanced: BloscCodec(cname='zstd', clevel=5)\n   # Maximum compression: GzipCodec(level=9)\n   ```\n\n4. **Storage Backend**: Match to environment\n   ```python\n   # Local: LocalStore (default)\n   # Cloud: fsspec URIs or FsspecStore + consolidated metadata\n   # Temporary: MemoryStore\n   ```\n\n5. **Sharding**: Use for large-scale datasets\n   ```python\n   # When you have millions of small chunks\n   shards=(10*chunk_size, 10*chunk_size)\n   ```\n\n6. **Parallel I\u002FO**: Use Dask for large operations\n   ```python\n   import dask.array as da\n   dask_array = da.from_zarr('data.zarr')\n   result = dask_array.compute(scheduler='threads', num_workers=8)\n   ```\n\n### Profiling and Debugging\n\n```python\n# Print detailed array information\nprint(z.info)\n\n# Output includes:\n# - Type, shape, chunks, dtype\n# - Serializer and compressors\n# - Storage size (compressed vs uncompressed)\n# - Storage location\n\n# Check storage size\nprint(f\"Compressed size: {z.nbytes_stored \u002F 1e6:.2f} MB\")\nprint(f\"Uncompressed size: {z.nbytes \u002F 1e6:.2f} MB\")\nprint(f\"Compression ratio: {z.nbytes \u002F z.nbytes_stored:.2f}x\")\n```\n\n## Common Patterns and Best Practices\n\n### Pattern: Time Series Data\n\n```python\n# Store time series with time as first dimension\n# This allows efficient appending of new time steps\nz = zarr.open('timeseries.zarr', mode='a',\n              shape=(0, 720, 1440),  # Start with 0 time steps\n              chunks=(1, 720, 1440),  # One time step per chunk\n              dtype='f4')\n\n# Append new time steps\nnew_data = np.random.random((1, 720, 1440))\nz.append(new_data, axis=0)\n```\n\n### Pattern: Large Matrix Operations\n\n```python\nimport dask.array as da\n\n# Create large matrix in Zarr\nz = zarr.open('matrix.zarr', mode='w',\n              shape=(100000, 100000),\n              chunks=(1000, 1000),\n              dtype='f8')\n\n# Use Dask for parallel computation\ndask_z = da.from_zarr('matrix.zarr')\nresult = (dask_z @ dask_z.T).compute()  # Parallel matrix multiply\n```\n\n### Pattern: Cloud-Native Workflow\n\n```python\nimport zarr\n\npath = \"s3:\u002F\u002Fmy-bucket\u002Fdata.zarr\"\nz = zarr.create_array(\n    store=path,\n    shape=(10000, 10000),\n    chunks=(500, 500),\n    dtype=\"f4\",\n    storage_options={\"anon\": False},\n)\nz[:] = data\n\nzarr.consolidate_metadata(path)\nz_read = zarr.open_consolidated(path, storage_options={\"anon\": False})\nsubset = z_read[0:100, 0:100]\n```\n\n### Pattern: Format Conversion\n\n```python\n# HDF5 to Zarr\nimport h5py\nimport zarr\n\nwith h5py.File('data.h5', 'r') as h5:\n    dataset = h5['dataset_name']\n    z = zarr.array(dataset[:],\n                   chunks=(1000, 1000),\n                   store='data.zarr')\n\n# NumPy to Zarr\nimport numpy as np\ndata = np.load('data.npy')\nz = zarr.array(data, chunks='auto', store='data.zarr')\n\n# Zarr to NetCDF (via Xarray)\nimport xarray as xr\nds = xr.open_zarr('data.zarr')\nds.to_netcdf('data.nc')\n```\n\n## Common Issues and Solutions\n\n### Issue: Slow Performance\n\n**Diagnosis**: Check chunk size and alignment\n```python\nprint(z.chunks)  # Are chunks appropriate size?\nprint(z.info)    # Check compression ratio\n```\n\n**Solutions**:\n- Increase chunk size to 1-10 MB\n- Align chunks with access pattern\n- Try different compression codecs\n- Use Dask for parallel operations\n\n### Issue: High Memory Usage\n\n**Cause**: Loading entire array or large chunks into memory\n\n**Solutions**:\n```python\n# Don't load entire array\n# Bad: data = z[:]\n# Good: Process in chunks\nfor i in range(0, z.shape[0], 1000):\n    chunk = z[i:i+1000, :]\n    process(chunk)\n\n# Or use Dask for automatic chunking\nimport dask.array as da\ndask_z = da.from_zarr('data.zarr')\nresult = dask_z.mean().compute()  # Processes in chunks\n```\n\n### Issue: Cloud Storage Latency\n\n**Solutions**:\n```python\n# 1. Consolidate metadata\nzarr.consolidate_metadata(store)\nz = zarr.open_consolidated(store)\n\n# 2. Use appropriate chunk sizes (5-100 MB for cloud)\nchunks = (2000, 2000)  # Larger chunks for cloud\n\n# 3. Enable sharding\nshards = (10000, 10000)  # Groups many chunks\n```\n\n### Issue: Concurrent Write Conflicts\n\n**Solution**: Design workflows so each process\u002Fthread writes to separate chunks. Zarr-Python 3 does not yet support `ThreadSynchronizer` \u002F `ProcessSynchronizer`; see `references\u002Fv3_migration.md`.\n\n## Additional Resources\n\n### Bundled references\n\n| File | Contents |\n|------|----------|\n| `references\u002Fapi_reference.md` | Function signatures, stores, codecs, indexing |\n| `references\u002Fv3_migration.md` | Zarr-Python 2→3 breaking changes and WIP features |\n\n### Official upstream\n\n- **Documentation**: https:\u002F\u002Fzarr.readthedocs.io\u002Fen\u002Fstable\u002F\n- **3.0 migration guide**: https:\u002F\u002Fzarr.readthedocs.io\u002Fen\u002Fstable\u002Fuser-guide\u002Fv3_migration\u002F\n- **Storage backends**: https:\u002F\u002Fzarr.readthedocs.io\u002Fen\u002Fstable\u002Fuser-guide\u002Fstorage\u002F\n- **Zarr specifications**: https:\u002F\u002Fzarr-specs.readthedocs.io\u002F\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fzarr-developers\u002Fzarr-python\n- **Developer chat**: https:\u002F\u002Fossci.zulipchat.com\u002F#narrow\u002Fchannel\u002F423692-Zarr-Python\n\n**Related libraries:** [Xarray](https:\u002F\u002Fdocs.xarray.dev\u002F), [Dask](https:\u002F\u002Fdocs.dask.org\u002F), [NumCodecs](https:\u002F\u002Fnumcodecs.readthedocs.io\u002F)\n\n",{"name":5,"license":24,"metadata":25,"description":7,"allowed-tools":28,"compatibility":29},"MIT license",{"version":26,"skill-author":27},"1.1","K-Dense Inc.","Read Write Edit Bash","Requires Python 3.12+ and zarr 3.x. Cloud I\u002FO needs zarr[remote] plus pinned s3fs or gcsfs. Legacy Zarr v2 workflows need exact 2.x pins on older Python.","main","8859218955d9504cd18b59e7892100718039d3b4","data\u002Fskills\u002Fk-dense-ai-scientific-agent-skills\u002Fzarr-python",1,"https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper","https:\u002F\u002Fgithub.com\u002Fom-scogo","SDE-1 @ Scogo ","2026-07-18 12:22:46",{"source":8,"skillName":5,"commands":39,"prompt":49,"zipAvailable":50},[40,43,46],{"tool":41,"command":42},"npx","npx skills add om-scogo\u002Fskillsh-scraper --skill zarr-python",{"tool":44,"command":45},"bunx","bunx skills add om-scogo\u002Fskillsh-scraper --skill zarr-python",{"tool":47,"command":48},"pnpm","pnpm dlx skills add om-scogo\u002Fskillsh-scraper --skill zarr-python","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper\u002Fblob\u002Fmain\u002Fdata\u002Fskills\u002Fk-dense-ai-scientific-agent-skills\u002Fzarr-python\u002FSkill.md",true,[52],{"path":53,"resourceType":54,"resourceTypeDesc":55,"sizeBytes":56,"sourceHtmlUrl":57},"metadata.json","REFERENCE","参考文档",498,"https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper\u002Fblob\u002Fmain\u002Fdata\u002Fskills\u002Fk-dense-ai-scientific-agent-skills\u002Fzarr-python\u002Fmetadata.json",[59,66,71,78,83,90],{"skillNo":60,"name":61,"nameZh":6,"description":62,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":63,"categories":64,"keywords":65,"publishedAt":19},"01M7D6MC9FAAEV4CMNYX8W7JTG","arxiv-search","Search arXiv physics, math, and computer science preprints using natural language queries. Powered by Valyu semantic search.","https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper\u002Fblob\u002Fmain\u002Fdata\u002Fskills\u002Fyorkeccak-scientific-skills\u002Farxiv-search\u002Fskills\u002Farxiv-search\u002FSKILL.md",[],[],{"skillNo":67,"name":61,"nameZh":6,"description":62,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":68,"categories":69,"keywords":70,"publishedAt":19},"01M7D6M85CKKN87TA98VCQV4H7","https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper\u002Fblob\u002Fmain\u002Fdata\u002Fskills\u002Fyorkeccak-scientific-skills\u002Farxiv-search\u002FSkill.md",[],[],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":75,"categories":76,"keywords":77,"publishedAt":19},"01M7D4ZTCS4DR3CE4JY06X84T2","working-with-legacy-code","Safely change and test untested codebases using Feathers' \"Working Effectively with Legacy Code\". Use when the user mentions \"legacy code\", \"no tests\", \"untested codebase\", \"how do I test this\", \"seams\", \"characterization tests\", \"golden master\", \"sprout method\", \"afraid to change this code\", \"monster method\", \"dependency breaking\", or \"inherited a messy codebase\". Also trigger when changing code without tests safely, getting a class under test when constructors, statics, or singletons block it, adding features to tangled modules, or planning incremental test coverage for an old codebase. Covers the legacy-code change algorithm, seams, characterization tests, sprout\u002Fwrap, and dependency-breaking techniques. For refactoring code that already has tests, see refactoring-patterns. For day-to-day code quality, see clean-code.","https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper\u002Fblob\u002Fmain\u002Fdata\u002Fskills\u002Fwondelai-skills\u002Fworking-with-legacy-code\u002Fworking-with-legacy-code\u002FSKILL.md",[],[],{"skillNo":79,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":80,"categories":81,"keywords":82,"publishedAt":19},"01M7D4ZHZKJD2RSNGAF63V14P4","https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper\u002Fblob\u002Fmain\u002Fdata\u002Fskills\u002Fwondelai-skills\u002Fworking-with-legacy-code\u002FSkill.md",[],[],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":19},"01M7D4ZB4CJBFW5JE67CPH2QRV","web-typography","Select, pair, and implement typefaces for web projects. Use when the user mentions \"font pairing\", \"which typeface\", \"line height\", \"responsive typography\", \"web font loading\", \"type hierarchy\", \"variable fonts\", \"FOUT\u002FFOIT\", \"typographic scale\", or \"the text is hard to read\". Also trigger when choosing between system fonts and web fonts, optimizing font-loading performance, or designing readable long-form content. Covers readability evaluation, CSS implementation, and performance optimization. For overall UI design systems, see refactoring-ui. For dramatic typographic experiences, see top-design.","https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper\u002Fblob\u002Fmain\u002Fdata\u002Fskills\u002Fwondelai-skills\u002Fweb-typography\u002Fweb-typography\u002FSKILL.md",[],[],{"skillNo":91,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":92,"categories":93,"keywords":94,"publishedAt":19},"01M7D4Z4MVYVY3NMJ18BNV6WNA","https:\u002F\u002Fgithub.com\u002Fom-scogo\u002Fskillsh-scraper\u002Fblob\u002Fmain\u002Fdata\u002Fskills\u002Fwondelai-skills\u002Fweb-typography\u002FSkill.md",[],[]]