[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6MAYJ99E23MC2HSFQ20WTCB":3,"skill-same:01M6MAYJ99E23MC2HSFQ20WTCB":98},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":53,"linkOnly":54,"markdownContent":55,"markdownBody":56,"frontMatter":57,"refName":64,"commitSha":65,"skillDirectory":66,"forksCount":67,"repositoryHtmlUrl":68,"repositoryHomepageUrl":69,"ownerHtmlUrl":70,"ownerBlogUrl":71,"ownerDescription":72,"updatedAt":73,"install":74,"resources":87},"01M6MAYJ99E23MC2HSFQ20WTCB","ray-data",null,"Scalable data processing for ML workloads. Streaming execution across CPU\u002FGPU, supports Parquet\u002FCSV\u002FJSON\u002Fimages. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use for batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipelines.","hashgraph-online\u002Fawesome-codex-plugins","hashgraph-online","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F160371962?v=4","01M6CNB6NRJVNNGN14TR01Q6D3","HOL (Hashgraph Online)","01M6JT1HSKKG9V68KQAQGBA7XP",1575,"MIT","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fray-data\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52],"Ray Data","data","Ray","images","for","loading","Ray Train","Batch Inference","ML Pipelines","Scalable","CPU","processing","preprocessing","etl pipelines","distributed computing","across","train","inference","pipelines","use","json","pandas","multimodal","streaming","batch","etl","pytorch","distributed","tensorflow","2026-11-10 10:12:30",false,"---\nname: ray-data\ndescription: Scalable data processing for ML workloads. Streaming execution across CPU\u002FGPU, supports Parquet\u002FCSV\u002FJSON\u002Fimages. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use for batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipelines.\nlicense: MIT\nmetadata:\n  version: 1.0.0\n  category: multimodal-and-emerging\n  maintainer: Kalaris Labs\n  tags: Data Processing, Ray Data, Distributed Computing, ML Pipelines, Batch Inference, ETL, Scalable, Ray, PyTorch, TensorFlow\n  dependencies: ray[data], pyarrow, pandas\n---\n\n# Ray Data - Scalable ML Data Processing\n\nDistributed data processing library for ML and AI workloads.\n\n## When to use Ray Data\n\n**Use Ray Data when:**\n- Processing large datasets (>100GB) for ML training\n- Need distributed data preprocessing across cluster\n- Building batch inference pipelines\n- Loading multi-modal data (images, audio, video)\n- Scaling data processing from laptop to cluster\n\n**Key features**:\n- **Streaming execution**: Process data larger than memory\n- **GPU support**: Accelerate transforms with GPUs\n- **Framework integration**: PyTorch, TensorFlow, HuggingFace\n- **Multi-modal**: Images, Parquet, CSV, JSON, audio, video\n\n**Use alternatives instead**:\n- **Pandas**: Small data (\u003C1GB) on single machine\n- **Dask**: Tabular data, SQL-like operations\n- **Spark**: Enterprise ETL, SQL queries\n\n## Quick start\n\n### Installation\n\n```bash\npip install -U 'ray[data]'\n```\n\n### Load and transform data\n\n```python\nimport ray\n\n# Read Parquet files\nds = ray.data.read_parquet(\"s3:\u002F\u002Fbucket\u002Fdata\u002F*.parquet\")\n\n# Transform data (lazy execution)\nds = ds.map_batches(lambda batch: {\"processed\": batch[\"text\"].str.lower()})\n\n# Consume data\nfor batch in ds.iter_batches(batch_size=100):\n    print(batch)\n```\n\n### Integration with Ray Train\n\n```python\nimport ray\nfrom ray.train import ScalingConfig\nfrom ray.train.torch import TorchTrainer\n\n# Create dataset\ntrain_ds = ray.data.read_parquet(\"s3:\u002F\u002Fbucket\u002Ftrain\u002F*.parquet\")\n\ndef train_func(config):\n    # Access dataset in training\n    train_ds = ray.train.get_dataset_shard(\"train\")\n\n    for epoch in range(10):\n        for batch in train_ds.iter_batches(batch_size=32):\n            # Train on batch\n            pass\n\n# Train with Ray\ntrainer = TorchTrainer(\n    train_func,\n    datasets={\"train\": train_ds},\n    scaling_config=ScalingConfig(num_workers=4, use_gpu=True)\n)\ntrainer.fit()\n```\n\n## Reading data\n\n### From cloud storage\n\n```python\nimport ray\n\n# Parquet (recommended for ML)\nds = ray.data.read_parquet(\"s3:\u002F\u002Fbucket\u002Fdata\u002F*.parquet\")\n\n# CSV\nds = ray.data.read_csv(\"s3:\u002F\u002Fbucket\u002Fdata\u002F*.csv\")\n\n# JSON\nds = ray.data.read_json(\"gs:\u002F\u002Fbucket\u002Fdata\u002F*.json\")\n\n# Images\nds = ray.data.read_images(\"s3:\u002F\u002Fbucket\u002Fimages\u002F\")\n```\n\n### From Python objects\n\n```python\n# From list\nds = ray.data.from_items([{\"id\": i, \"value\": i * 2} for i in range(1000)])\n\n# From range\nds = ray.data.range(1000000)  # Synthetic data\n\n# From pandas\nimport pandas as pd\ndf = pd.DataFrame({\"col1\": [1, 2, 3], \"col2\": [4, 5, 6]})\nds = ray.data.from_pandas(df)\n```\n\n## Transformations\n\n### Map batches (vectorized)\n\n```python\n# Batch transformation (fast)\ndef process_batch(batch):\n    batch[\"doubled\"] = batch[\"value\"] * 2\n    return batch\n\nds = ds.map_batches(process_batch, batch_size=1000)\n```\n\n### Row transformations\n\n```python\n# Row-by-row (slower)\ndef process_row(row):\n    row[\"squared\"] = row[\"value\"] ** 2\n    return row\n\nds = ds.map(process_row)\n```\n\n### Filter\n\n```python\n# Filter rows\nds = ds.filter(lambda row: row[\"value\"] > 100)\n```\n\n### Group by and aggregate\n\n```python\n# Group by column\nds = ds.groupby(\"category\").count()\n\n# Custom aggregation\nds = ds.groupby(\"category\").map_groups(lambda group: {\"sum\": group[\"value\"].sum()})\n```\n\n## GPU-accelerated transforms\n\n```python\n# Use GPU for preprocessing\ndef preprocess_images_gpu(batch):\n    import torch\n    images = torch.tensor(batch[\"image\"]).cuda()\n    # GPU preprocessing\n    processed = images * 255\n    return {\"processed\": processed.cpu().numpy()}\n\nds = ds.map_batches(\n    preprocess_images_gpu,\n    batch_size=64,\n    num_gpus=1  # Request GPU\n)\n```\n\n## Writing data\n\n```python\n# Write to Parquet\nds.write_parquet(\"s3:\u002F\u002Fbucket\u002Foutput\u002F\")\n\n# Write to CSV\nds.write_csv(\"output\u002F\")\n\n# Write to JSON\nds.write_json(\"output\u002F\")\n```\n\n## Performance optimization\n\n### Repartition\n\n```python\n# Control parallelism\nds = ds.repartition(100)  # 100 blocks for 100-core cluster\n```\n\n### Batch size tuning\n\n```python\n# Larger batches = faster vectorized ops\nds.map_batches(process_fn, batch_size=10000)  # vs batch_size=100\n```\n\n### Streaming execution\n\n```python\n# Process data larger than memory\nds = ray.data.read_parquet(\"s3:\u002F\u002Fhuge-dataset\u002F\")\nfor batch in ds.iter_batches(batch_size=1000):\n    process(batch)  # Streamed, not loaded to memory\n```\n\n## Common patterns\n\n### Batch inference\n\n```python\nimport ray\n\n# Load model\ndef load_model():\n    # Load once per worker\n    return MyModel()\n\n# Inference function\nclass BatchInference:\n    def __init__(self):\n        self.model = load_model()\n\n    def __call__(self, batch):\n        predictions = self.model(batch[\"input\"])\n        return {\"prediction\": predictions}\n\n# Run distributed inference\nds = ray.data.read_parquet(\"s3:\u002F\u002Fdata\u002F\")\npredictions = ds.map_batches(BatchInference, batch_size=32, num_gpus=1)\npredictions.write_parquet(\"s3:\u002F\u002Foutput\u002F\")\n```\n\n### Data preprocessing pipeline\n\n```python\n# Multi-step pipeline\nds = (\n    ray.data.read_parquet(\"s3:\u002F\u002Fraw\u002F\")\n    .map_batches(clean_data)\n    .map_batches(tokenize)\n    .map_batches(augment)\n    .write_parquet(\"s3:\u002F\u002Fprocessed\u002F\")\n)\n```\n\n## Integration with ML frameworks\n\n### PyTorch\n\n```python\n# Convert to PyTorch\ntorch_ds = ds.to_torch(label_column=\"label\", batch_size=32)\n\nfor batch in torch_ds:\n    # batch is dict with tensors\n    inputs, labels = batch[\"features\"], batch[\"label\"]\n```\n\n### TensorFlow\n\n```python\n# Convert to TensorFlow\ntf_ds = ds.to_tf(feature_columns=[\"image\"], label_column=\"label\", batch_size=32)\n\nfor features, labels in tf_ds:\n    # Train model\n    pass\n```\n\n## Supported data formats\n\n| Format | Read | Write | Use Case |\n|--------|------|-------|----------|\n| Parquet | ✅ | ✅ | ML data (recommended) |\n| CSV | ✅ | ✅ | Tabular data |\n| JSON | ✅ | ✅ | Semi-structured |\n| Images | ✅ | ❌ | Computer vision |\n| NumPy | ✅ | ✅ | Arrays |\n| Pandas | ✅ | ❌ | DataFrames |\n\n## Performance benchmarks\n\n**Scaling** (processing 100GB data):\n- 1 node (16 cores): ~30 minutes\n- 4 nodes (64 cores): ~8 minutes\n- 16 nodes (256 cores): ~2 minutes\n\n**GPU acceleration** (image preprocessing):\n- CPU only: 1,000 images\u002Fsec\n- 1 GPU: 5,000 images\u002Fsec\n- 4 GPUs: 18,000 images\u002Fsec\n\n## Use cases\n\n**Production deployments**:\n- **Pinterest**: Last-mile data processing for model training\n- **ByteDance**: Scaling offline inference with multi-modal LLMs\n- **Spotify**: ML platform for batch inference\n\n## References\n\n- **[Transformations Guide](references\u002Ftransformations.md)** - Map, filter, groupby operations\n- **[Integration Guide](references\u002Fintegration.md)** - Ray Train, PyTorch, TensorFlow\n\n## Resources\n\n- **Docs**: https:\u002F\u002Fdocs.ray.io\u002Fen\u002Flatest\u002Fdata\u002Fdata.html\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fray-project\u002Fray ⭐ 36,000+\n- **Version**: Ray 2.40.0+\n- **Examples**: https:\u002F\u002Fdocs.ray.io\u002Fen\u002Flatest\u002Fdata\u002Fexamples\u002Foverview.html\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `ray-train`: Distributed training orchestration across clusters.\n- `polars`: High-performance DataFrame library for Python ETL, analytics, and pandas migration.\n- `dask`: Distributed computing for larger-than-RAM pandas\u002FNumPy workflows.\n","# Ray Data - Scalable ML Data Processing\n\nDistributed data processing library for ML and AI workloads.\n\n## When to use Ray Data\n\n**Use Ray Data when:**\n- Processing large datasets (>100GB) for ML training\n- Need distributed data preprocessing across cluster\n- Building batch inference pipelines\n- Loading multi-modal data (images, audio, video)\n- Scaling data processing from laptop to cluster\n\n**Key features**:\n- **Streaming execution**: Process data larger than memory\n- **GPU support**: Accelerate transforms with GPUs\n- **Framework integration**: PyTorch, TensorFlow, HuggingFace\n- **Multi-modal**: Images, Parquet, CSV, JSON, audio, video\n\n**Use alternatives instead**:\n- **Pandas**: Small data (\u003C1GB) on single machine\n- **Dask**: Tabular data, SQL-like operations\n- **Spark**: Enterprise ETL, SQL queries\n\n## Quick start\n\n### Installation\n\n```bash\npip install -U 'ray[data]'\n```\n\n### Load and transform data\n\n```python\nimport ray\n\n# Read Parquet files\nds = ray.data.read_parquet(\"s3:\u002F\u002Fbucket\u002Fdata\u002F*.parquet\")\n\n# Transform data (lazy execution)\nds = ds.map_batches(lambda batch: {\"processed\": batch[\"text\"].str.lower()})\n\n# Consume data\nfor batch in ds.iter_batches(batch_size=100):\n    print(batch)\n```\n\n### Integration with Ray Train\n\n```python\nimport ray\nfrom ray.train import ScalingConfig\nfrom ray.train.torch import TorchTrainer\n\n# Create dataset\ntrain_ds = ray.data.read_parquet(\"s3:\u002F\u002Fbucket\u002Ftrain\u002F*.parquet\")\n\ndef train_func(config):\n    # Access dataset in training\n    train_ds = ray.train.get_dataset_shard(\"train\")\n\n    for epoch in range(10):\n        for batch in train_ds.iter_batches(batch_size=32):\n            # Train on batch\n            pass\n\n# Train with Ray\ntrainer = TorchTrainer(\n    train_func,\n    datasets={\"train\": train_ds},\n    scaling_config=ScalingConfig(num_workers=4, use_gpu=True)\n)\ntrainer.fit()\n```\n\n## Reading data\n\n### From cloud storage\n\n```python\nimport ray\n\n# Parquet (recommended for ML)\nds = ray.data.read_parquet(\"s3:\u002F\u002Fbucket\u002Fdata\u002F*.parquet\")\n\n# CSV\nds = ray.data.read_csv(\"s3:\u002F\u002Fbucket\u002Fdata\u002F*.csv\")\n\n# JSON\nds = ray.data.read_json(\"gs:\u002F\u002Fbucket\u002Fdata\u002F*.json\")\n\n# Images\nds = ray.data.read_images(\"s3:\u002F\u002Fbucket\u002Fimages\u002F\")\n```\n\n### From Python objects\n\n```python\n# From list\nds = ray.data.from_items([{\"id\": i, \"value\": i * 2} for i in range(1000)])\n\n# From range\nds = ray.data.range(1000000)  # Synthetic data\n\n# From pandas\nimport pandas as pd\ndf = pd.DataFrame({\"col1\": [1, 2, 3], \"col2\": [4, 5, 6]})\nds = ray.data.from_pandas(df)\n```\n\n## Transformations\n\n### Map batches (vectorized)\n\n```python\n# Batch transformation (fast)\ndef process_batch(batch):\n    batch[\"doubled\"] = batch[\"value\"] * 2\n    return batch\n\nds = ds.map_batches(process_batch, batch_size=1000)\n```\n\n### Row transformations\n\n```python\n# Row-by-row (slower)\ndef process_row(row):\n    row[\"squared\"] = row[\"value\"] ** 2\n    return row\n\nds = ds.map(process_row)\n```\n\n### Filter\n\n```python\n# Filter rows\nds = ds.filter(lambda row: row[\"value\"] > 100)\n```\n\n### Group by and aggregate\n\n```python\n# Group by column\nds = ds.groupby(\"category\").count()\n\n# Custom aggregation\nds = ds.groupby(\"category\").map_groups(lambda group: {\"sum\": group[\"value\"].sum()})\n```\n\n## GPU-accelerated transforms\n\n```python\n# Use GPU for preprocessing\ndef preprocess_images_gpu(batch):\n    import torch\n    images = torch.tensor(batch[\"image\"]).cuda()\n    # GPU preprocessing\n    processed = images * 255\n    return {\"processed\": processed.cpu().numpy()}\n\nds = ds.map_batches(\n    preprocess_images_gpu,\n    batch_size=64,\n    num_gpus=1  # Request GPU\n)\n```\n\n## Writing data\n\n```python\n# Write to Parquet\nds.write_parquet(\"s3:\u002F\u002Fbucket\u002Foutput\u002F\")\n\n# Write to CSV\nds.write_csv(\"output\u002F\")\n\n# Write to JSON\nds.write_json(\"output\u002F\")\n```\n\n## Performance optimization\n\n### Repartition\n\n```python\n# Control parallelism\nds = ds.repartition(100)  # 100 blocks for 100-core cluster\n```\n\n### Batch size tuning\n\n```python\n# Larger batches = faster vectorized ops\nds.map_batches(process_fn, batch_size=10000)  # vs batch_size=100\n```\n\n### Streaming execution\n\n```python\n# Process data larger than memory\nds = ray.data.read_parquet(\"s3:\u002F\u002Fhuge-dataset\u002F\")\nfor batch in ds.iter_batches(batch_size=1000):\n    process(batch)  # Streamed, not loaded to memory\n```\n\n## Common patterns\n\n### Batch inference\n\n```python\nimport ray\n\n# Load model\ndef load_model():\n    # Load once per worker\n    return MyModel()\n\n# Inference function\nclass BatchInference:\n    def __init__(self):\n        self.model = load_model()\n\n    def __call__(self, batch):\n        predictions = self.model(batch[\"input\"])\n        return {\"prediction\": predictions}\n\n# Run distributed inference\nds = ray.data.read_parquet(\"s3:\u002F\u002Fdata\u002F\")\npredictions = ds.map_batches(BatchInference, batch_size=32, num_gpus=1)\npredictions.write_parquet(\"s3:\u002F\u002Foutput\u002F\")\n```\n\n### Data preprocessing pipeline\n\n```python\n# Multi-step pipeline\nds = (\n    ray.data.read_parquet(\"s3:\u002F\u002Fraw\u002F\")\n    .map_batches(clean_data)\n    .map_batches(tokenize)\n    .map_batches(augment)\n    .write_parquet(\"s3:\u002F\u002Fprocessed\u002F\")\n)\n```\n\n## Integration with ML frameworks\n\n### PyTorch\n\n```python\n# Convert to PyTorch\ntorch_ds = ds.to_torch(label_column=\"label\", batch_size=32)\n\nfor batch in torch_ds:\n    # batch is dict with tensors\n    inputs, labels = batch[\"features\"], batch[\"label\"]\n```\n\n### TensorFlow\n\n```python\n# Convert to TensorFlow\ntf_ds = ds.to_tf(feature_columns=[\"image\"], label_column=\"label\", batch_size=32)\n\nfor features, labels in tf_ds:\n    # Train model\n    pass\n```\n\n## Supported data formats\n\n| Format | Read | Write | Use Case |\n|--------|------|-------|----------|\n| Parquet | ✅ | ✅ | ML data (recommended) |\n| CSV | ✅ | ✅ | Tabular data |\n| JSON | ✅ | ✅ | Semi-structured |\n| Images | ✅ | ❌ | Computer vision |\n| NumPy | ✅ | ✅ | Arrays |\n| Pandas | ✅ | ❌ | DataFrames |\n\n## Performance benchmarks\n\n**Scaling** (processing 100GB data):\n- 1 node (16 cores): ~30 minutes\n- 4 nodes (64 cores): ~8 minutes\n- 16 nodes (256 cores): ~2 minutes\n\n**GPU acceleration** (image preprocessing):\n- CPU only: 1,000 images\u002Fsec\n- 1 GPU: 5,000 images\u002Fsec\n- 4 GPUs: 18,000 images\u002Fsec\n\n## Use cases\n\n**Production deployments**:\n- **Pinterest**: Last-mile data processing for model training\n- **ByteDance**: Scaling offline inference with multi-modal LLMs\n- **Spotify**: ML platform for batch inference\n\n## References\n\n- **[Transformations Guide](references\u002Ftransformations.md)** - Map, filter, groupby operations\n- **[Integration Guide](references\u002Fintegration.md)** - Ray Train, PyTorch, TensorFlow\n\n## Resources\n\n- **Docs**: https:\u002F\u002Fdocs.ray.io\u002Fen\u002Flatest\u002Fdata\u002Fdata.html\n- **GitHub**: https:\u002F\u002Fgithub.com\u002Fray-project\u002Fray ⭐ 36,000+\n- **Version**: Ray 2.40.0+\n- **Examples**: https:\u002F\u002Fdocs.ray.io\u002Fen\u002Flatest\u002Fdata\u002Fexamples\u002Foverview.html\n\n## Agent operating procedure\n\n1. **Check the environment.** Confirm GPU memory, model checkpoint and licence, framework versions and input formats.\n2. **Pin down the inputs.** Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.\n3. **Run a small version first.** Run inference on a single sample at low resolution or short length first.\n4. **Execute the full task** using the instructions and references above.\n5. **Validate the result.** Inspect outputs qualitatively and with task metrics; check licence restrictions for generated content.\n6. **Report.** State what was run (versions, commands, parameters), what was checked, and what is still uncertain.\n\n| If this happens | Do this |\n|---|---|\n| Out-of-memory on large inputs | Reduce resolution or sequence length, use half precision, or tile the input. |\n| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (`help()`, `--help`, official docs), adapt, and tell the user. Never invent an API. |\n| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |\n\n**Integrity rules**\n\n- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.\n- Check model and dataset licences before commercial or published use.\n- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.\n- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.\n\n## Related skills\n\n- `ray-train`: Distributed training orchestration across clusters.\n- `polars`: High-performance DataFrame library for Python ETL, analytics, and pandas migration.\n- `dask`: Distributed computing for larger-than-RAM pandas\u002FNumPy workflows.\n",{"name":5,"license":15,"metadata":58,"description":7},{"tags":59,"version":60,"category":61,"maintainer":62,"dependencies":63},"Data Processing, Ray Data, Distributed Computing, ML Pipelines, Batch Inference, ETL, Scalable, Ray, PyTorch, TensorFlow","1.0.0","multimodal-and-emerging","Kalaris Labs","ray[data], pyarrow, pandas","main","2b63f856ce19c2a797929cacabd9e0fc936812fa","plugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fray-data",491,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins","https:\u002F\u002Fhol.org\u002Fplugins\u002Fbest-codex-plugins","https:\u002F\u002Fgithub.com\u002Fhashgraph-online","https:\u002F\u002Fhol.org","Open-source standards and developer tools for interoperable AI agents: discovery, identity, communication, payments, privacy and security.","2026-11-14 14:06:13",{"source":8,"skillName":5,"commands":75,"prompt":85,"zipAvailable":86},[76,79,82],{"tool":77,"command":78},"npx","npx skills add hashgraph-online\u002Fawesome-codex-plugins --skill ray-data",{"tool":80,"command":81},"bunx","bunx skills add hashgraph-online\u002Fawesome-codex-plugins --skill ray-data",{"tool":83,"command":84},"pnpm","pnpm dlx skills add hashgraph-online\u002Fawesome-codex-plugins --skill ray-data","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fray-data\u002FSKILL.md",true,[88,94],{"path":89,"resourceType":90,"resourceTypeDesc":91,"sizeBytes":92,"sourceHtmlUrl":93},"references\u002Fintegration.md","REFERENCE","参考文档",1851,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fray-data\u002Freferences\u002Fintegration.md",{"path":95,"resourceType":90,"resourceTypeDesc":91,"sizeBytes":96,"sourceHtmlUrl":97},"references\u002Ftransformations.md",1664,"https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002FKalarisLabs\u002Fresearch-agent-skills\u002Fskills\u002Fray-data\u002Freferences\u002Ftransformations.md",[99,108,115,122,149,174],{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":103,"licenseFileUrl":6,"sourceHtmlUrl":104,"categories":105,"keywords":106,"publishedAt":107},"01M746ZWACVKMZ1BJXJZHYJSSX","migrate-to-e2e","Moves an existing Playwright, Cypress, Selenium, Appium, Detox, Maestro, or TesterArmy suite to e2e one slice at a time, with the old suite running beside the new one until each ported test is proven. Keeps a MIGRATION.md progress file so any later session resumes where the last stopped. Optionally plants a defect to show both suites catch it. Use for \u002Fmigrate-to-e2e, \"migrate our tests to e2e\", \"port these Maestro flows\", \"move our TesterArmy tests\", or when a project has an e2e config and an older suite still running.","Apache-2.0","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fmigrate-to-e2e\u002FSKILL.md",[],[],"2026-11-11 15:11:14",{"skillNo":109,"name":110,"nameZh":6,"description":111,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":103,"licenseFileUrl":6,"sourceHtmlUrl":112,"categories":113,"keywords":114,"publishedAt":107},"01M746ZPTAX1Q0FBKAB1CYND7E","e2e","Writes, runs, and debugs end-to-end tests for web and mobile apps with e2e, the agentic test runner, which mixes agent.act goals with exact locator assertions on a Playwright browser or an iOS or Android device. Also drives a project's running app live over the e2e MCP server to look at a screen, check a UI change, take a screenshot or record a video, and try a locator; and runs bug bashes, parallel e2e explore runs with each bug proven by a repro test. Use when a project depends on e2e or has an e2e.config.ts, when asked for end-to-end, browser, mobile, or agentic UI tests, to check a change in the running app, to QA or bug bash an app or branch, or when an e2e run fails.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fe2e\u002FSKILL.md",[],[],{"skillNo":116,"name":117,"nameZh":6,"description":118,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":103,"licenseFileUrl":6,"sourceHtmlUrl":119,"categories":120,"keywords":121,"publishedAt":107},"01M746ZJN784HVHMDP9MQGCSGG","create-verification-skill","Generate a project-local verify-\u003Capp> skill on top of e2e, so any coding agent can launch the app, check it is worth driving, exercise a feature the way a user does, keep evidence, and bug bash it. Writes the skill, a feature map with one explore charter per feature, and proves it once. Use for \u002Fcreate-verification-skill, \"make a verification skill for this repo\", or when a project has no scripted way to prove UI behavior.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftester-army\u002Fe2e\u002Fskills\u002Fcreate-verification-skill\u002FSKILL.md",[],[],{"skillNo":123,"name":124,"nameZh":6,"description":125,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":103,"licenseFileUrl":6,"sourceHtmlUrl":126,"categories":127,"keywords":128,"publishedAt":53},"01M716A6ETNB9B1SQ3GXZC75VR","deja-search","deja-vu memory — search the user's past AI coding sessions with the deja CLI. Holds what they already settled and never wrote into the repo (naming rules, banned or already-tried tools, required checks before merge, preferred values, who owns what). Use before your first edit in a task, before you call a change done or ready to merge, before you recommend a tool, library, design or reviewer, before re-debugging an error that may already be solved, when they say 'didn't we fix this before', 'what did we decide about X' or 'I already have that', and before telling them something on this machine does not exist.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-search\u002FSKILL.md",[],[129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148],"search","deja search","never","error","ready","memory","re","decide","fix","design","debugging","when","you","done","reviewer","io","change","cli","merge","past",{"skillNo":150,"name":151,"nameZh":6,"description":152,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":103,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":155,"publishedAt":53},"01M716A3J2W5SENSE9VGMXB375","deja-reflect","End-of-session reflection and memory curation with deja-vu. Review what was decided, what commands broke and what repaired them, and record durable project rules or decisions with deja remember so the next session starts with them.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Fvshulcz\u002Fdeja-vu\u002Fskills\u002Fdeja-reflect\u002FSKILL.md",[],[156,157,158,144,159,160,161,162,163,164,165,166,167,168,134,169,170,171,172,173],"deja reflect","reflect","review","with","record","github","next","https","node","Rules","decisions","brew","remember","project rules","project","session","end","reflection",{"skillNo":175,"name":176,"nameZh":6,"description":177,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":103,"licenseFileUrl":6,"sourceHtmlUrl":178,"categories":179,"keywords":191,"publishedAt":53},"01M715T3RR4BAYSR57HPPTJ0PF","revid","Make, edit, export and post AI videos with Revid (revid.ai) through the Revid MCP server. Use whenever the user wants a short-form or long-form video made (from an idea, a script, a URL, an article, a PDF, a tweet, a product page, their own clips, or a song), wants to find what is going viral on TikTok, YouTube, Instagram or X before making something, wants to edit or re-export an existing Revid project, generate a standalone AI image or clip, make a series episode, or publish and schedule a finished video to connected social accounts. Covers setup (hosted MCP with OAuth, or an API key), the 36 tools, credits, polling and the rules for publishing. Routes to the more specific revid-* skills when one fits.","https:\u002F\u002Fgithub.com\u002Fhashgraph-online\u002Fawesome-codex-plugins\u002Fblob\u002Fmain\u002Fplugins\u002Ftmakersas\u002Frevid-skills\u002Fskills\u002Frevid\u002FSKILL.md",[180,185,188],{"categoryNo":181,"categoryName":182,"taxonomyType":183,"taxonomyTypeDesc":184},"01JBSEEDCAT000000000000016","图像与多媒体","SCENARIO","使用场景",{"categoryNo":186,"categoryName":187,"taxonomyType":183,"taxonomyTypeDesc":184},"01JBSEEDCAT000000000000023","Skill与智能体开发",{"categoryNo":189,"categoryName":190,"taxonomyType":183,"taxonomyTypeDesc":184},"01JBSEEDCAT000000000000010","后端与接口",[176,192,193,194,195,196,197,170,198,135,199,200,201,202,140,203,204,205,206,207],"URL","Video","short-form","standalone","PDF","viral","tiktok","short","social","MCP","song","image","mcp server","schedule","api","publish"]