[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6ACJA72KJRAM6F7WHMX01PN":3,"skill-same:01M6ACJA72KJRAM6F7WHMX01PN":69},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":40,"linkOnly":41,"markdownContent":42,"markdownBody":43,"frontMatter":44,"refName":46,"commitSha":47,"skillDirectory":48,"forksCount":49,"repositoryHtmlUrl":50,"repositoryHomepageUrl":51,"ownerHtmlUrl":52,"ownerBlogUrl":53,"ownerDescription":6,"updatedAt":54,"install":55,"resources":68},"01M6ACJA72KJRAM6F7WHMX01PN","review-comet-ffi-pr",null,"Use when reviewing a DataFusion Comet pull request that crosses the JVM\u002Fnative boundary, touching Arrow C Data or C Stream interface code, batch export and import, CometExecIterator, ScanExec, NativeUtil, CometVector subclasses, or jni_api. Load alongside review-comet-pr.","apache\u002Fdatafusion-comet","apache","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F47359?v=4","01M6AC83E6K4FDKK7HZ8WNMQKE","The Apache Software Foundation","01M6ACHX42TPNCR8TQJEAR8J5J",1320,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-ffi-pr\u002FSKILL.md",[],[19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39],"pr","review","review comet ffi pr","ffi","when","request","load","number","interface","native","import","jvm","use","api","boundary","datafusion","batch","arrow","pull","code","pull request","2026-11-10 10:12:30",false,"---\nname: review-comet-ffi-pr\ndescription: Use when reviewing a DataFusion Comet pull request that crosses the JVM\u002Fnative boundary, touching Arrow C Data or C Stream interface code, batch export and import, CometExecIterator, ScanExec, NativeUtil, CometVector subclasses, or jni_api. Load alongside review-comet-pr.\nargument-hint: \u003Cpr-number>\n---\n\n\u003C!--\nLicensed to the Apache Software Foundation (ASF) under one\nor more contributor license agreements.  See the NOTICE file\ndistributed with this work for additional information\nregarding copyright ownership.  The ASF licenses this file\nto you under the Apache License, Version 2.0 (the\n\"License\"); you may not use this file except in compliance\nwith the License.  You may obtain a copy of the License at\n\n  http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0\n\nUnless required by applicable law or agreed to in writing,\nsoftware distributed under the License is distributed on an\n\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, either express or implied.  See the License for the\nspecific language governing permissions and limitations\nunder the License.\n-->\n\nFFI-specific review for Comet PR #$ARGUMENTS.\n\n**REQUIRED BACKGROUND:** Use `review-comet-pr` for PR metadata, existing comments, CI, the review\nbar, and the output format. This skill only covers the JVM\u002Fnative boundary.\n\nBugs in this area do not produce wrong answers, they produce segfaults, leaks, and use-after-free\nunder load, often only on one platform or only when an operator buffers batches. Review it with\nthat in mind.\n\n## Read the Contributor Guide First\n\n| Doc                                                  | What you need from it                                                       |\n| ---------------------------------------------------- | --------------------------------------------------------------------------- |\n| `docs\u002Fsource\u002Fcontributor-guide\u002Fffi.md`               | Both data-flow directions, ownership rules, lifecycle, offset normalization |\n| `docs\u002Fsource\u002Fcontributor-guide\u002Fmemory_management.md` | The \"Crossing the FFI boundary\" section: who is charged for a batch's bytes |\n\nRead `ffi.md` in full before the diff. The two directions have different ownership semantics and\nreviewing one with the other's mental model is the most common way to miss a bug.\n\n## The Two Directions\n\n| Direction                           | Mechanism                                  | Who owns the data                                                  |\n| ----------------------------------- | ------------------------------------------ | ------------------------------------------------------------------ |\n| JVM to native (`ScanExec`)          | Arrow C **Stream**, one per partition      | Native takes ownership by reference count when it imports a batch  |\n| Native to JVM (`CometExecIterator`) | Arrow C **Data**, one array pair per batch | Native allocates, JVM holds pointers and must `close()` to release |\n\n## 1. Ownership and Lifetime\n\n- [ ] **JVM to native: no defensive deep copies.** The C Stream transfers ownership by reference\n      count, so native can buffer imported batches in `SortExec` or the shuffle writer without\n      copying. A new `.clone()` of the data, a `MutableArrayData` copy, or a \"to be safe\" deep copy\n      on this path is a real throughput cost. Ask what it is protecting against.\n- [ ] **JVM to native: dropping the reader is the release.** When `ScanExec` drops its\n      `ArrowArrayStreamReader`, the stream's release callback fires synchronously back into the\n      JVM and closes the `ArrowReader` and its `VectorSchemaRoot`. Anything that extends the\n      reader's lifetime, stores it somewhere longer-lived, or drops it early changes when JVM\n      off-heap buffers are freed.\n- [ ] **JVM to native: buffering pins JVM memory.** An operator that holds many imported batches\n      keeps the corresponding JVM-side off-heap buffers alive. A change that makes an operator\n      buffer more is also a memory change.\n- [ ] **Native to JVM: every export needs a matching close.** Native allocates, the JVM wraps the\n      pointers in `ArrowBuf`s, and the bytes are freed only when the JVM calls `close()`. Trace the\n      new export to the `close()` that releases it, including on the exception path.\n- [ ] **Release callbacks run on the error path too.** A batch exported and then abandoned because\n      the query failed still has to be released. Check the failure and cancellation paths, not just\n      the happy path.\n- [ ] **No unwinding across `extern \"C\"`.** A Rust panic crossing the FFI boundary is undefined\n      behavior. New `#[no_mangle] extern \"system\"` entry points must not let a panic escape, and\n      must not `unwrap()` on anything an input can make fail.\n- [ ] **Null and error checks on every pointer received from the other side.**\n\n## 2. The Stream Design\n\nThe JVM exports each per-partition iterator **once** as an `ArrowArrayStream`, and native pulls\nevery batch through the stream's `get_next` callback. There is no per-batch JNI call and no\nper-column FFI export on this path.\n\nA change that reintroduces per-batch or per-column export on the JVM-to-native path is a\nperformance regression even if it is correct. Flag it and ask why the stream could not carry it.\n\nThe reader implementations in `CometNativeArrowSource.scala` are `RowArrowReader` for\n`Iterator[InternalRow]`, `SparkColumnarArrowReader` for a non-Arrow `ColumnarBatch`, and\n`ColumnarBatchArrowReader` for an Arrow-backed `ColumnarBatch`, which transfers `VectorSchemaRoot`\nownership. A new input shape needs a reader, not a special case elsewhere.\n\n## 3. Vector Types and Export Dispatch\n\n`NativeUtil.exportBatch()` matches on the column type and has exactly two cases. A `CometVector`\nexports its underlying `FieldVector` through `Data.exportVector`, passing the dictionary provider\nonly when the field carries a dictionary. Spark's own `ConstantColumnVector` is materialized into a\nfresh Arrow vector first, because native takes Arrow arrays only. Anything else throws\n`\"Comet execution only takes Arrow Arrays\"`.\n\nThe `CometVector` hierarchy is the abstract `CometVector`, the abstract `CometDecodedVector` under\nit, and the concrete `CometPlainVector`, `CometDictionaryVector`, `CometListVector`,\n`CometMapVector`, and `CometStructVector`. All of them export through the one `CometVector` case, so\na new subclass needs no new case as long as `getValueVector` returns an Arrow vector.\n\n- [ ] A new column type that is **not** a `CometVector` needs its own case, and needs to say who\n      owns the vector it materializes. The `ConstantColumnVector` case allocates a new Arrow vector\n      per batch, which is a real cost on a hot path.\n- [ ] A new `CometVector` subclass whose `getValueVector` is synthesized rather than owned has a\n      defined lifetime relative to the export.\n- [ ] Import is symmetric. `ScanExec::pull_next` runs every imported column through\n      `import_column`, which decodes invalid UTF-8 to Spark's rendering and otherwise keeps the\n      imported buffers. A PR that adds a column type has to say what that step does to it.\n- [ ] The row-count check still holds. `exportBatch` requires every column to report the same value\n      count and throws otherwise, which is the guard that catches a vector exported at the wrong\n      length.\n\n## 4. Alignment, Offsets, and Schema\n\n- [ ] **JVM to native: arrow-rs does the alignment.** Java's allocator hands back `Decimal128`\n      buffers at 8-byte rather than 16-byte alignment. Since arrow 59, `from_ffi` and\n      `from_ffi_and_data_type` realign them on import\n      ([arrow-rs#10030](https:\u002F\u002Fgithub.com\u002Fapache\u002Farrow-rs\u002Fpull\u002F10030)), so `ScanExec` reads the\n      stream with the stock `ArrowArrayStreamReader`. The `realigns_under_aligned_decimal128` test\n      in `scan.rs` guards this. A PR that downgrades arrow, or that imports through anything other\n      than those two functions, must keep that test passing.\n- [ ] **Native to JVM: exported offsets must be zero.** Arrow Java ignores `ArrowArray.offset` at\n      every level on import, so every array native exports to the JVM first goes through\n      `zero_offsets` (in `native\u002Fcommon\u002Fsrc\u002Fffi_offsets.rs`). `move_to_spark` applies it to\n      executed batches and decoded shuffle blocks, and `JvmScalarUdfExpr` to the inputs of the\n      JVM UDF bridge. A new export path has to call it too, or a sliced boolean, top-level or\n      nested, reaches the JVM misaligned\n      ([#6288](https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fissues\u002F6288)).\n- [ ] **Schema reconciliation stays truthful.** `CometArrowStream.reconcileStreamSchema` advertises\n      the stream's schema from the actual `CometVector` types in the first batch rather than the\n      consumer's Spark-declared types, so that the cast in native `build_record_batch` fires. A PR\n      that changes either side needs to change both, and a new \"just declare the Spark type\" path\n      quietly reintroduces the drift this exists to handle.\n- [ ] **Dictionaries are decoded on the JVM.** No input stream carries a dictionary:\n      `ColumnarBatchArrowReader` decodes dictionary-encoded columns before export, and\n      `reconcileStreamSchema` advertises the value type. A new reader that exports a dictionary\n      would reach `ScanExec` as is, and only the cast in `build_record_batch` would unpack it.\n- [ ] **Timestamps cross unconverted.** Both directions pass the raw microseconds. JVM producers\n      label `TimestampType` with `CometArrowStream.NATIVE_TIMEZONE`, which is `\"UTC\"`, and a new\n      producer must use it too rather than the session timezone. A change that shifts values by a\n      timezone at the boundary gives wrong answers. See \"How Comet represents timestamps\" in\n      `docs\u002Fsource\u002Fcontributor-guide\u002Ftimezones.md`.\n\n## 5. Memory Accounting\n\nAn FFI change is usually also a memory change, and the two are easy to review separately and miss\nthe interaction. Imported JVM buffers come from `CometArrowAllocator`, which is an unbounded\n`RootAllocator(Long.MaxValue)` that no budget sees. Exported native batches are usually no longer\npool-reserved by the time the JVM receives them, but they stay resident until the JVM closes them.\n\nIf the PR changes how long either side holds a batch, use `review-comet-memory-pr` as well.\n\n## 6. Tests\n\nFFI bugs rarely reproduce in a small unit test. Ask what evidence the PR offers:\n\n- Does an existing suite exercise the new path with a non-trivial number of batches, rather than\n  one batch that happens to work?\n- Are nested and dictionary-encoded types covered, since they take different import paths?\n- Are Decimal128 columns covered, since they need an alignment that Java's allocator does not\n  guarantee?\n- On the native to JVM side, are sliced inputs covered, since Arrow Java ignores a non-zero\n  offset?\n- For a lifecycle change, is there a test that closes or cancels early?\n- Rust tests in `native\u002Fcore` need `DYLD_LIBRARY_PATH=$JAVA_HOME\u002Flib\u002Fserver` on macOS and\n  `LD_LIBRARY_PATH` on Linux. If the PR adds one that needs the JVM, check CI actually runs it.\n\n## 7. Does the PR Make `ffi.md` Stale?\n\n`ffi.md` documents mechanism, so mechanism changes invalidate it. Check specifically:\n\n- The **Memory Ownership Rules** tables at the end, one per direction. A change to who owns or who\n  frees rewrites a row.\n- The **JVM to Native** and **Native to JVM** architecture diagrams, if a stage is added, removed,\n  or renamed.\n- The list of `ArrowReader` implementations, if the PR adds an input shape.\n- The **Array Offsets** section, if the PR adds a native to JVM export path or changes what\n  `zero_offsets` does.\n- The **Schema Reconciliation** and **Ownership and Lifecycle** paragraphs, which state invariants\n  rather than describing code. These are the easiest to leave quietly wrong.\n- Class and file paths named in the prose, if the PR moves anything.\n\nCheck also whether the \"Crossing the FFI boundary\" section of `memory_management.md` still holds,\nsince it describes which operators reserve for imported batches by name.\n","\u003C!--\nLicensed to the Apache Software Foundation (ASF) under one\nor more contributor license agreements.  See the NOTICE file\ndistributed with this work for additional information\nregarding copyright ownership.  The ASF licenses this file\nto you under the Apache License, Version 2.0 (the\n\"License\"); you may not use this file except in compliance\nwith the License.  You may obtain a copy of the License at\n\n  http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0\n\nUnless required by applicable law or agreed to in writing,\nsoftware distributed under the License is distributed on an\n\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, either express or implied.  See the License for the\nspecific language governing permissions and limitations\nunder the License.\n-->\n\nFFI-specific review for Comet PR #$ARGUMENTS.\n\n**REQUIRED BACKGROUND:** Use `review-comet-pr` for PR metadata, existing comments, CI, the review\nbar, and the output format. This skill only covers the JVM\u002Fnative boundary.\n\nBugs in this area do not produce wrong answers, they produce segfaults, leaks, and use-after-free\nunder load, often only on one platform or only when an operator buffers batches. Review it with\nthat in mind.\n\n## Read the Contributor Guide First\n\n| Doc                                                  | What you need from it                                                       |\n| ---------------------------------------------------- | --------------------------------------------------------------------------- |\n| `docs\u002Fsource\u002Fcontributor-guide\u002Fffi.md`               | Both data-flow directions, ownership rules, lifecycle, offset normalization |\n| `docs\u002Fsource\u002Fcontributor-guide\u002Fmemory_management.md` | The \"Crossing the FFI boundary\" section: who is charged for a batch's bytes |\n\nRead `ffi.md` in full before the diff. The two directions have different ownership semantics and\nreviewing one with the other's mental model is the most common way to miss a bug.\n\n## The Two Directions\n\n| Direction                           | Mechanism                                  | Who owns the data                                                  |\n| ----------------------------------- | ------------------------------------------ | ------------------------------------------------------------------ |\n| JVM to native (`ScanExec`)          | Arrow C **Stream**, one per partition      | Native takes ownership by reference count when it imports a batch  |\n| Native to JVM (`CometExecIterator`) | Arrow C **Data**, one array pair per batch | Native allocates, JVM holds pointers and must `close()` to release |\n\n## 1. Ownership and Lifetime\n\n- [ ] **JVM to native: no defensive deep copies.** The C Stream transfers ownership by reference\n      count, so native can buffer imported batches in `SortExec` or the shuffle writer without\n      copying. A new `.clone()` of the data, a `MutableArrayData` copy, or a \"to be safe\" deep copy\n      on this path is a real throughput cost. Ask what it is protecting against.\n- [ ] **JVM to native: dropping the reader is the release.** When `ScanExec` drops its\n      `ArrowArrayStreamReader`, the stream's release callback fires synchronously back into the\n      JVM and closes the `ArrowReader` and its `VectorSchemaRoot`. Anything that extends the\n      reader's lifetime, stores it somewhere longer-lived, or drops it early changes when JVM\n      off-heap buffers are freed.\n- [ ] **JVM to native: buffering pins JVM memory.** An operator that holds many imported batches\n      keeps the corresponding JVM-side off-heap buffers alive. A change that makes an operator\n      buffer more is also a memory change.\n- [ ] **Native to JVM: every export needs a matching close.** Native allocates, the JVM wraps the\n      pointers in `ArrowBuf`s, and the bytes are freed only when the JVM calls `close()`. Trace the\n      new export to the `close()` that releases it, including on the exception path.\n- [ ] **Release callbacks run on the error path too.** A batch exported and then abandoned because\n      the query failed still has to be released. Check the failure and cancellation paths, not just\n      the happy path.\n- [ ] **No unwinding across `extern \"C\"`.** A Rust panic crossing the FFI boundary is undefined\n      behavior. New `#[no_mangle] extern \"system\"` entry points must not let a panic escape, and\n      must not `unwrap()` on anything an input can make fail.\n- [ ] **Null and error checks on every pointer received from the other side.**\n\n## 2. The Stream Design\n\nThe JVM exports each per-partition iterator **once** as an `ArrowArrayStream`, and native pulls\nevery batch through the stream's `get_next` callback. There is no per-batch JNI call and no\nper-column FFI export on this path.\n\nA change that reintroduces per-batch or per-column export on the JVM-to-native path is a\nperformance regression even if it is correct. Flag it and ask why the stream could not carry it.\n\nThe reader implementations in `CometNativeArrowSource.scala` are `RowArrowReader` for\n`Iterator[InternalRow]`, `SparkColumnarArrowReader` for a non-Arrow `ColumnarBatch`, and\n`ColumnarBatchArrowReader` for an Arrow-backed `ColumnarBatch`, which transfers `VectorSchemaRoot`\nownership. A new input shape needs a reader, not a special case elsewhere.\n\n## 3. Vector Types and Export Dispatch\n\n`NativeUtil.exportBatch()` matches on the column type and has exactly two cases. A `CometVector`\nexports its underlying `FieldVector` through `Data.exportVector`, passing the dictionary provider\nonly when the field carries a dictionary. Spark's own `ConstantColumnVector` is materialized into a\nfresh Arrow vector first, because native takes Arrow arrays only. Anything else throws\n`\"Comet execution only takes Arrow Arrays\"`.\n\nThe `CometVector` hierarchy is the abstract `CometVector`, the abstract `CometDecodedVector` under\nit, and the concrete `CometPlainVector`, `CometDictionaryVector`, `CometListVector`,\n`CometMapVector`, and `CometStructVector`. All of them export through the one `CometVector` case, so\na new subclass needs no new case as long as `getValueVector` returns an Arrow vector.\n\n- [ ] A new column type that is **not** a `CometVector` needs its own case, and needs to say who\n      owns the vector it materializes. The `ConstantColumnVector` case allocates a new Arrow vector\n      per batch, which is a real cost on a hot path.\n- [ ] A new `CometVector` subclass whose `getValueVector` is synthesized rather than owned has a\n      defined lifetime relative to the export.\n- [ ] Import is symmetric. `ScanExec::pull_next` runs every imported column through\n      `import_column`, which decodes invalid UTF-8 to Spark's rendering and otherwise keeps the\n      imported buffers. A PR that adds a column type has to say what that step does to it.\n- [ ] The row-count check still holds. `exportBatch` requires every column to report the same value\n      count and throws otherwise, which is the guard that catches a vector exported at the wrong\n      length.\n\n## 4. Alignment, Offsets, and Schema\n\n- [ ] **JVM to native: arrow-rs does the alignment.** Java's allocator hands back `Decimal128`\n      buffers at 8-byte rather than 16-byte alignment. Since arrow 59, `from_ffi` and\n      `from_ffi_and_data_type` realign them on import\n      ([arrow-rs#10030](https:\u002F\u002Fgithub.com\u002Fapache\u002Farrow-rs\u002Fpull\u002F10030)), so `ScanExec` reads the\n      stream with the stock `ArrowArrayStreamReader`. The `realigns_under_aligned_decimal128` test\n      in `scan.rs` guards this. A PR that downgrades arrow, or that imports through anything other\n      than those two functions, must keep that test passing.\n- [ ] **Native to JVM: exported offsets must be zero.** Arrow Java ignores `ArrowArray.offset` at\n      every level on import, so every array native exports to the JVM first goes through\n      `zero_offsets` (in `native\u002Fcommon\u002Fsrc\u002Fffi_offsets.rs`). `move_to_spark` applies it to\n      executed batches and decoded shuffle blocks, and `JvmScalarUdfExpr` to the inputs of the\n      JVM UDF bridge. A new export path has to call it too, or a sliced boolean, top-level or\n      nested, reaches the JVM misaligned\n      ([#6288](https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fissues\u002F6288)).\n- [ ] **Schema reconciliation stays truthful.** `CometArrowStream.reconcileStreamSchema` advertises\n      the stream's schema from the actual `CometVector` types in the first batch rather than the\n      consumer's Spark-declared types, so that the cast in native `build_record_batch` fires. A PR\n      that changes either side needs to change both, and a new \"just declare the Spark type\" path\n      quietly reintroduces the drift this exists to handle.\n- [ ] **Dictionaries are decoded on the JVM.** No input stream carries a dictionary:\n      `ColumnarBatchArrowReader` decodes dictionary-encoded columns before export, and\n      `reconcileStreamSchema` advertises the value type. A new reader that exports a dictionary\n      would reach `ScanExec` as is, and only the cast in `build_record_batch` would unpack it.\n- [ ] **Timestamps cross unconverted.** Both directions pass the raw microseconds. JVM producers\n      label `TimestampType` with `CometArrowStream.NATIVE_TIMEZONE`, which is `\"UTC\"`, and a new\n      producer must use it too rather than the session timezone. A change that shifts values by a\n      timezone at the boundary gives wrong answers. See \"How Comet represents timestamps\" in\n      `docs\u002Fsource\u002Fcontributor-guide\u002Ftimezones.md`.\n\n## 5. Memory Accounting\n\nAn FFI change is usually also a memory change, and the two are easy to review separately and miss\nthe interaction. Imported JVM buffers come from `CometArrowAllocator`, which is an unbounded\n`RootAllocator(Long.MaxValue)` that no budget sees. Exported native batches are usually no longer\npool-reserved by the time the JVM receives them, but they stay resident until the JVM closes them.\n\nIf the PR changes how long either side holds a batch, use `review-comet-memory-pr` as well.\n\n## 6. Tests\n\nFFI bugs rarely reproduce in a small unit test. Ask what evidence the PR offers:\n\n- Does an existing suite exercise the new path with a non-trivial number of batches, rather than\n  one batch that happens to work?\n- Are nested and dictionary-encoded types covered, since they take different import paths?\n- Are Decimal128 columns covered, since they need an alignment that Java's allocator does not\n  guarantee?\n- On the native to JVM side, are sliced inputs covered, since Arrow Java ignores a non-zero\n  offset?\n- For a lifecycle change, is there a test that closes or cancels early?\n- Rust tests in `native\u002Fcore` need `DYLD_LIBRARY_PATH=$JAVA_HOME\u002Flib\u002Fserver` on macOS and\n  `LD_LIBRARY_PATH` on Linux. If the PR adds one that needs the JVM, check CI actually runs it.\n\n## 7. Does the PR Make `ffi.md` Stale?\n\n`ffi.md` documents mechanism, so mechanism changes invalidate it. Check specifically:\n\n- The **Memory Ownership Rules** tables at the end, one per direction. A change to who owns or who\n  frees rewrites a row.\n- The **JVM to Native** and **Native to JVM** architecture diagrams, if a stage is added, removed,\n  or renamed.\n- The list of `ArrowReader` implementations, if the PR adds an input shape.\n- The **Array Offsets** section, if the PR adds a native to JVM export path or changes what\n  `zero_offsets` does.\n- The **Schema Reconciliation** and **Ownership and Lifecycle** paragraphs, which state invariants\n  rather than describing code. These are the easiest to leave quietly wrong.\n- Class and file paths named in the prose, if the PR moves anything.\n\nCheck also whether the \"Crossing the FFI boundary\" section of `memory_management.md` still holds,\nsince it describes which operators reserve for imported batches by name.\n",{"name":5,"description":7,"argument-hint":45},"\u003Cpr-number>","main","7deeba21a42e4a9cd34f43e7ed6e960e4e851b26",".ai\u002Fskills\u002Freview-comet-ffi-pr",415,"https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet","https:\u002F\u002Fdatafusion.apache.org\u002Fcomet","https:\u002F\u002Fgithub.com\u002Fapache","https:\u002F\u002Fwww.apache.org\u002F","2026-11-11 09:39:28",{"source":8,"skillName":5,"commands":56,"prompt":66,"zipAvailable":67},[57,60,63],{"tool":58,"command":59},"npx","npx skills add apache\u002Fdatafusion-comet --skill review-comet-ffi-pr",{"tool":61,"command":62},"bunx","bunx skills add apache\u002Fdatafusion-comet --skill review-comet-ffi-pr",{"tool":64,"command":65},"pnpm","pnpm dlx skills add apache\u002Fdatafusion-comet --skill review-comet-ffi-pr","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-ffi-pr\u002FSKILL.md",true,[],[70,99,123,136,153,164],{"skillNo":71,"name":72,"nameZh":6,"description":73,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":74,"categories":75,"keywords":81,"publishedAt":40},"01M6ACJR50ZS7F1QX58JEKCGVM","wire-datafusion-function","Use when wiring an existing DataFusion or datafusion-spark function into Comet for a Spark expression. Identifies the right wiring pattern (one-line passthrough, datafusion-spark UDF registration, or custom serde with input massaging \u002F restrictions), applies the Scala serde, registers the UDF in jni_api when needed, and adds SQL file tests. Assumes the function already exists upstream — if not, switch to `implement-comet-expression`.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Fwire-datafusion-function\u002FSKILL.md",[76],{"categoryNo":77,"categoryName":78,"taxonomyType":79,"taxonomyTypeDesc":80},"01JBSEEDCAT000000000000010","后端与接口","SCENARIO","使用场景",[34,82,83,84,85,86,87,88,89,90,91,92,93,31,94,23,95,32,96,97,98],"wire datafusion function","sql","scala","implement","spark","registration","switch","for","upstream","custom","expression","tests","Existing","file","wiring","with","name",{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":40},"01M6ACJPDDPND6FHE2PC1CJ4D3","suggest-native-expression","Use when picking the next Spark expression to implement natively in Comet, or when asked whether a specific codegen-dispatched expression is worth a native Rust implementation. Scores the candidate on how likely a native path can be 100% Spark-compatible and how much throughput or allocation it would save versus the JVM codegen dispatcher, records the verdict in the per-expression audit log so disqualified candidates are not re-litigated, and files a GitHub issue for the recommendation.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Fsuggest-native-expression\u002FSKILL.md",[],[106,92,28,107,108,109,85,86,110,111,23,112,113,30,114,115,116,117,118,119,120,121,122],"suggest native expression","dispatcher","github","rust","re","files","candidate","log","next","how","audit","audit log","allocation","issue","throughput","path","save",{"skillNo":124,"name":125,"nameZh":6,"description":126,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":127,"categories":128,"keywords":129,"publishedAt":40},"01M6ACJMHPEN7E76VPKC3P4X6C","review-comet-shuffle-pr","Use when reviewing a DataFusion Comet pull request that touches native or JVM columnar shuffle, the shuffle writers and readers, partitioning, the Arrow IPC block format, shuffle compression, or the Celeborn integration. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-shuffle-pr\u002FSKILL.md",[],[130,19,20,37,39,34,36,31,25,23,131,26,30,28,24,132,133,134,135],"review comet shuffle pr","ipc","format","integration","block","compression",{"skillNo":137,"name":138,"nameZh":6,"description":139,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":140,"categories":141,"keywords":142,"publishedAt":40},"01M6ACJHX8BES7ZNNRNVCQ4QWE","review-comet-pr","Use when reviewing a DataFusion Comet pull request. Covers the workflow that applies to every PR and routes to the area-specific review skills for expressions, FFI, memory management, shuffle, Iceberg writes, and documentation. Provides guidance to a human reviewer rather than posting comments.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-pr\u002FSKILL.md",[],[19,20,143,37,144,39,145,34,146,147,23,26,148,149,24,150,89,31,151,152],"review comet pr","iceberg","memory","workflow","documentation","reviewer","human","posting","review skills","comments",{"skillNo":154,"name":155,"nameZh":6,"description":156,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":157,"categories":158,"keywords":159,"publishedAt":40},"01M6ACJFM0QS61Z1RW516HXN2M","review-comet-memory-pr","Use when reviewing a DataFusion Comet pull request that touches memory pools, memory reservations, spilling, the Spark TaskMemoryManager bridge, allocators, or memory-related configuration. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-memory-pr\u002FSKILL.md",[],[19,145,160,20,37,39,34,86,161,162,31,25,23,26,24,163],"review comet memory pr","bridge","reservations","configuration",{"skillNo":165,"name":166,"nameZh":6,"description":167,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":168,"categories":169,"keywords":170,"publishedAt":40},"01M6ACJCV64V0FZPMVXDE2K3DX","review-comet-iceberg-write-pr","Use when reviewing a DataFusion Comet pull request that touches Iceberg writes, the split-operator write plan (IcebergWriteStrategy, IcebergWriteExec, IcebergCommitExec), the native Iceberg writer (CometIcebergWriteExec, iceberg_write.rs, iceberg_partition_path.rs), its eligibility gate (CometIcebergNativeWrite), or the iceberg-rust pin. Load alongside review-comet-pr.","https:\u002F\u002Fgithub.com\u002Fapache\u002Fdatafusion-comet\u002Fblob\u002Fmain\u002F.ai\u002Fskills\u002Freview-comet-iceberg-write-pr\u002FSKILL.md",[],[19,144,171,20,172,37,39,34,109,173,174,175,31,25,176,23,26,177,178,28],"review comet iceberg write pr","write","writer","write plan","partition","pin","split","plan"]