[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6X0KKEEFVCG0CVB9R60PYVZ":3,"skill-same:01M6X0KKEEFVCG0CVB9R60PYVZ":84},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":28,"publishedAt":44,"linkOnly":45,"markdownContent":46,"markdownBody":47,"frontMatter":48,"refName":57,"commitSha":58,"skillDirectory":59,"forksCount":14,"repositoryHtmlUrl":60,"repositoryHomepageUrl":61,"ownerHtmlUrl":62,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":63,"install":64,"resources":77},"01M6X0KKEEFVCG0CVB9R60PYVZ","pandas-polars",null,"DataFrame operations with pandas and polars — groupby, joins, reshaping, performance. Use when manipulating tabular data, choosing between pandas and polars, optimizing DataFrame code, or translating between the two libraries.","projectious-work\u002Fprocesskit","projectious-work","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F263889726?v=4","01M6N97TJJ118WQ35CHRV8RD0E","projectious.work","01M6N97TPWJPF0BMKYQ4PBY5W8",0,"MIT","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Farchive\u002Fprocesskit-install-20261101\u002Fcontext\u002Ftemplates\u002Fprocesskit\u002Fv0.22.0\u002Fcontext\u002Fskills\u002Fdata-ai\u002Fpandas-polars\u002FSKILL.md",[18,23],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000013","分析与可视化","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":26,"taxonomyTypeDesc":27},"01JBSEEDCAT000000000000033","数据分析师","PROFESSION","适用职业",[29,30,31,32,33,34,35,36,37,38,39,40,41,42,43],"polars","pandas","pandas polars","performance","libraries","code","dataframe","data","work","operations","use","with","v1","tabular","when","2026-11-10 10:12:30",false,"---\nname: pandas-polars\ndescription: |\n  DataFrame operations with pandas and polars — groupby, joins, reshaping, performance. Use when manipulating tabular data, choosing between pandas and polars, optimizing DataFrame code, or translating between the two libraries.\nmetadata:\n  processkit:\n    apiVersion: processkit.projectious.work\u002Fv1\n    id: SKILL-pandas-polars\n    version: \"1.1.0\"\n    created: 2026-04-06T00:00:00Z\n    category: data-ai\n---\n\n# Pandas and Polars\n\n## Intro\n\nPandas is the mature, broad-ecosystem default for exploratory work\nunder ~1 GB; polars is faster, leaner, and lazy-evaluated for\nproduction pipelines and larger data. Both read the same formats —\nwhen in doubt, match what the project already uses.\n\n## Overview\n\n### Choosing between them\n\n- **Pandas** — mature ecosystem, broader library compatibility\n  (sklearn, plotly), familiar API, best for EDA and \u003C 1 GB datasets\n- **Polars** — faster execution, lower memory, lazy evaluation,\n  better for production pipelines and 1 GB+ datasets\n- Polars has no index — use explicit columns instead\n- If the project already uses one, prefer consistency over switching\n\n### I\u002FO\n\nPrefer Parquet over CSV for performance and type preservation.\nSpecify `dtype` \u002F `schema` on read to enforce types upfront. For\nlarge CSVs, use chunked reads (pandas) or lazy scans (polars).\nAlways specify `parse_dates` or date columns explicitly —\nauto-detection is unreliable. Use `usecols` \u002F `columns` to read only\nthe columns you need.\n\n### Selection and filtering\n\nSelect columns by name, not position. In polars, use expressions:\n`df.select(pl.col(\"name\"), pl.col(\"age\"))`. In pandas, use `.loc[]`\nfor label-based and `.iloc[]` for position-based. Chain filters\nwith `&` and `|`, wrapping conditions in parentheses. Polars\n`filter()` is preferred over boolean indexing for clarity.\n\n### GroupBy and aggregation\n\nAlways name aggregated columns explicitly:\n\n```python\n# Polars\ndf.group_by(\"category\").agg(pl.col(\"price\").mean().alias(\"avg_price\"))\n\n# Pandas\ndf.groupby(\"category\")[\"price\"].mean().reset_index(name=\"avg_price\")\n```\n\nUse multiple aggregations in one pass for efficiency. Polars\nsupports `over()` for window functions without groupby. Avoid\niterating over groups — use vectorized aggregations.\n\n### Joins and merges\n\nSpecify join type explicitly: `inner`, `left`, `outer`, `cross`,\n`anti`, `semi`. Use `on=` for same-named keys, `left_on=`\u002F\n`right_on=` otherwise. Check for duplicates before joining to avoid\nrow explosion. Polars anti-join is efficient for \"rows not in\nother table.\" Use `validate=\"m:1\"` in pandas to catch unexpected\nduplicates.\n\n### Reshaping\n\nPivot to wide format for summary tables; melt to long for analysis\nand plotting:\n\n```python\n# Polars\ndf.pivot(on=\"category\", index=\"date\", values=\"sales\")\n\n# Pandas\ndf.pivot_table(index=\"date\", columns=\"category\", values=\"sales\", aggfunc=\"sum\")\n```\n\nUse `stack()`\u002F`unstack()` in pandas for multi-level index reshaping.\n\n### Missing data, strings, datetimes\n\nCheck missing with `df.null_count()` (polars) or `df.isna().sum()`\n(pandas). Fill with `fill_null` \u002F `fillna`, or forward\u002Fbackward\nfill. Drop only when appropriate. Strip whitespace and lowercase\nstrings early. Parse dates on read with explicit format. Always be\nexplicit about timezones — naive datetimes cause subtle bugs.\n\n### Performance\n\nUse lazy evaluation in polars: `df.lazy().filter(...).select(...).\ncollect()`. Polars optimizes the query plan, pushing filters and\nprojections down. In pandas, avoid `apply()` with Python functions —\nuse vectorized operations. Use categorical dtype for low-cardinality\nstrings. Profile with `%timeit` and check memory with\n`df.estimated_size()` (polars).\n\n## Gotchas\n\nAgent-specific failure modes — provider-neutral pause-and-self-check items:\n\n- **Using `apply()` with a Python function in pandas for performance-sensitive code.** `apply()` drops to a Python loop, which is 10–100x slower than vectorized operations. Use built-in string methods, `map()`, `np.where()`, or Polars expressions instead.\n- **Joining without checking for duplicates first.** A join on a key that has duplicates in either table causes a row explosion (M × N rows for M and N matching rows). Check `df[\"key\"].is_unique` before joining; validate with `validate=\"m:1\"` in pandas.\n- **Auto-detecting date formats or dtypes on read.** Pandas' auto-detection of dates and dtypes is unreliable across file versions and locales. Always specify `dtype=` and `parse_dates=` (pandas) or `schema=` (polars) explicitly on read. A column that auto-parsed as `datetime64` in development may parse as `object` on a slightly different CSV in production.\n- **Ignoring naive datetimes and timezone bugs.** A naive datetime (no timezone) compared to a tz-aware datetime raises an error in polars and silently produces wrong results in some pandas operations. Always be explicit about timezones — parse with an explicit `time_zone=` and convert to UTC at the boundary.\n- **Using polars with pandas `apply()` mental model.** Polars expressions are lazy and operate on the whole column — there is no row-by-row loop. Translating pandas `apply()` to polars `map_elements()` (element-wise Python) negates polars' performance advantage. Rewrite in terms of polars expression API.\n- **Not using lazy evaluation in polars for large datasets.** Calling `pl.read_csv(...)` eagerly loads the entire file into memory. Use `pl.scan_csv(...).lazy()` and chain `.filter()`, `.select()`, `.collect()` so polars can push projections and filters down and avoid materializing intermediate results.\n- **Storing multiple logical tables in one DataFrame.** Mixing two different entities (orders and line items) in one DataFrame with repeated parent-level columns produces confusing GroupBy results and joins. Separate into clean normalized DataFrames and join when needed.\n\n## Full reference\n\n### API comparison — reading data\n\n```python\n# Pandas\ndf = pd.read_csv(\"data.csv\", usecols=[\"a\", \"b\"], dtype={\"a\": \"int64\"})\ndf = pd.read_parquet(\"data.parquet\", columns=[\"a\", \"b\"])\n\n# Polars\ndf = pl.read_csv(\"data.csv\", columns=[\"a\", \"b\"], schema={\"a\": pl.Int64})\ndf = pl.read_parquet(\"data.parquet\", columns=[\"a\", \"b\"])\n\n# Polars lazy (recommended for large files)\nlf = pl.scan_csv(\"data.csv\")\ndf = lf.filter(...).select(...).collect()\n```\n\n### Selecting columns\n\n```python\n# Pandas\ndf[[\"name\", \"age\"]]\ndf.loc[:, \"name\":\"age\"]\n\n# Polars\ndf.select(\"name\", \"age\")\ndf.select(pl.col(\"name\"), pl.col(\"age\"))\ndf.select(pl.col(\"^sales_.*$\"))  # regex column selection\n```\n\n### Filtering rows\n\n```python\n# Pandas\ndf[df[\"age\"] > 30]\ndf[(df[\"age\"] > 30) & (df[\"city\"] == \"Berlin\")]\ndf.query(\"age > 30 and city == 'Berlin'\")\n\n# Polars\ndf.filter(pl.col(\"age\") > 30)\ndf.filter((pl.col(\"age\") > 30) & (pl.col(\"city\") == \"Berlin\"))\n```\n\n### Adding \u002F transforming columns\n\n```python\n# Pandas\ndf[\"total\"] = df[\"price\"] * df[\"qty\"]\ndf = df.assign(total=lambda d: d[\"price\"] * d[\"qty\"])\n\n# Polars\ndf = df.with_columns((pl.col(\"price\") * pl.col(\"qty\")).alias(\"total\"))\ndf = df.with_columns(\n    pl.col(\"name\").str.to_uppercase().alias(\"name_upper\"),\n    (pl.col(\"price\") * 1.1).alias(\"price_with_tax\"),\n)\n```\n\n### GroupBy and aggregation\n\n```python\n# Pandas\ndf.groupby(\"category\").agg(\n    avg_price=(\"price\", \"mean\"),\n    total_qty=(\"qty\", \"sum\"),\n    count=(\"id\", \"count\"),\n).reset_index()\n\n# Polars\ndf.group_by(\"category\").agg(\n    pl.col(\"price\").mean().alias(\"avg_price\"),\n    pl.col(\"qty\").sum().alias(\"total_qty\"),\n    pl.col(\"id\").count().alias(\"count\"),\n)\n```\n\n### Window functions\n\n```python\n# Pandas\ndf[\"rank\"] = df.groupby(\"category\")[\"price\"].rank(ascending=False)\ndf[\"avg_in_group\"] = df.groupby(\"category\")[\"price\"].transform(\"mean\")\n\n# Polars\ndf = df.with_columns(\n    pl.col(\"price\").rank(descending=True).over(\"category\").alias(\"rank\"),\n    pl.col(\"price\").mean().over(\"category\").alias(\"avg_in_group\"),\n)\n```\n\n### Joins\n\n```python\n# Pandas\npd.merge(left, right, on=\"id\", how=\"left\")\npd.merge(left, right, left_on=\"user_id\", right_on=\"id\", how=\"inner\")\n\n# Polars\nleft.join(right, on=\"id\", how=\"left\")\nleft.join(right, left_on=\"user_id\", right_on=\"id\", how=\"inner\")\n\n# Anti-join (rows in left NOT in right)\nleft[~left[\"id\"].isin(right[\"id\"])]            # pandas\nleft.join(right, on=\"id\", how=\"anti\")          # polars\n```\n\n### Pivot and melt\n\n```python\n# Pivot — Pandas\ndf.pivot_table(index=\"date\", columns=\"product\", values=\"sales\", aggfunc=\"sum\")\n# Pivot — Polars\ndf.pivot(on=\"product\", index=\"date\", values=\"sales\", aggregate_function=\"sum\")\n\n# Melt — Pandas\ndf.melt(id_vars=[\"date\"], value_vars=[\"product_a\", \"product_b\"],\n        var_name=\"product\", value_name=\"sales\")\n# Melt — Polars\ndf.unpivot(index=\"date\", on=[\"product_a\", \"product_b\"],\n           variable_name=\"product\", value_name=\"sales\")\n```\n\n### Missing data\n\n```python\n# Pandas\ndf.isna().sum()\ndf.fillna({\"price\": 0, \"name\": \"Unknown\"})\ndf.dropna(subset=[\"critical_col\"])\ndf[\"value\"].ffill()\n\n# Polars\ndf.null_count()\ndf.with_columns(\n    pl.col(\"price\").fill_null(0),\n    pl.col(\"name\").fill_null(\"Unknown\"),\n)\ndf.drop_nulls(subset=[\"critical_col\"])\ndf.with_columns(pl.col(\"value\").forward_fill())\n```\n\n### When to use which\n\n| Scenario | Recommendation |\n|---|---|\n| Exploratory data analysis | Pandas (Jupyter, wider ecosystem) |\n| Dataset > 1 GB | Polars (lower memory, parallel) |\n| Production data pipeline | Polars (lazy eval, deterministic) |\n| sklearn \u002F statsmodels integration | Pandas (native support) |\n| Streaming \u002F chunked processing | Polars lazy or pandas chunks |\n| Simple one-off scripts | Either — use what the team knows |\n| Multi-threaded workloads | Polars (releases GIL, built-in parallelism) |\n\n### Worked examples\n\n- **Large CSV → filter → group stats:** Polars lazy mode, scan the\n  CSV, filter before collect, group with multiple aggregations in\n  one `.agg()`, sort, write Parquet.\n- **Pandas → polars conversion:** Replace `.loc[]` with\n  `filter()`\u002F`select()`, convert `groupby().agg()` to polars\n  `group_by().agg()` with `pl.col()` expressions, drop index-based\n  ops, add `.lazy()`\u002F`.collect()` for optimization.\n- **Messy dataset cleanup:** Drop exact duplicates, identify\n  high-null columns, fill or drop based on context, lowercase and\n  strip strings, parse dates with explicit format, validate value\n  ranges via filter expressions.\n\n### References\n\n- [Pandas Documentation](https:\u002F\u002Fpandas.pydata.org\u002Fdocs\u002F)\n- [Polars User Guide](https:\u002F\u002Fdocs.pola.rs\u002F)\n","# Pandas and Polars\n\n## Intro\n\nPandas is the mature, broad-ecosystem default for exploratory work\nunder ~1 GB; polars is faster, leaner, and lazy-evaluated for\nproduction pipelines and larger data. Both read the same formats —\nwhen in doubt, match what the project already uses.\n\n## Overview\n\n### Choosing between them\n\n- **Pandas** — mature ecosystem, broader library compatibility\n  (sklearn, plotly), familiar API, best for EDA and \u003C 1 GB datasets\n- **Polars** — faster execution, lower memory, lazy evaluation,\n  better for production pipelines and 1 GB+ datasets\n- Polars has no index — use explicit columns instead\n- If the project already uses one, prefer consistency over switching\n\n### I\u002FO\n\nPrefer Parquet over CSV for performance and type preservation.\nSpecify `dtype` \u002F `schema` on read to enforce types upfront. For\nlarge CSVs, use chunked reads (pandas) or lazy scans (polars).\nAlways specify `parse_dates` or date columns explicitly —\nauto-detection is unreliable. Use `usecols` \u002F `columns` to read only\nthe columns you need.\n\n### Selection and filtering\n\nSelect columns by name, not position. In polars, use expressions:\n`df.select(pl.col(\"name\"), pl.col(\"age\"))`. In pandas, use `.loc[]`\nfor label-based and `.iloc[]` for position-based. Chain filters\nwith `&` and `|`, wrapping conditions in parentheses. Polars\n`filter()` is preferred over boolean indexing for clarity.\n\n### GroupBy and aggregation\n\nAlways name aggregated columns explicitly:\n\n```python\n# Polars\ndf.group_by(\"category\").agg(pl.col(\"price\").mean().alias(\"avg_price\"))\n\n# Pandas\ndf.groupby(\"category\")[\"price\"].mean().reset_index(name=\"avg_price\")\n```\n\nUse multiple aggregations in one pass for efficiency. Polars\nsupports `over()` for window functions without groupby. Avoid\niterating over groups — use vectorized aggregations.\n\n### Joins and merges\n\nSpecify join type explicitly: `inner`, `left`, `outer`, `cross`,\n`anti`, `semi`. Use `on=` for same-named keys, `left_on=`\u002F\n`right_on=` otherwise. Check for duplicates before joining to avoid\nrow explosion. Polars anti-join is efficient for \"rows not in\nother table.\" Use `validate=\"m:1\"` in pandas to catch unexpected\nduplicates.\n\n### Reshaping\n\nPivot to wide format for summary tables; melt to long for analysis\nand plotting:\n\n```python\n# Polars\ndf.pivot(on=\"category\", index=\"date\", values=\"sales\")\n\n# Pandas\ndf.pivot_table(index=\"date\", columns=\"category\", values=\"sales\", aggfunc=\"sum\")\n```\n\nUse `stack()`\u002F`unstack()` in pandas for multi-level index reshaping.\n\n### Missing data, strings, datetimes\n\nCheck missing with `df.null_count()` (polars) or `df.isna().sum()`\n(pandas). Fill with `fill_null` \u002F `fillna`, or forward\u002Fbackward\nfill. Drop only when appropriate. Strip whitespace and lowercase\nstrings early. Parse dates on read with explicit format. Always be\nexplicit about timezones — naive datetimes cause subtle bugs.\n\n### Performance\n\nUse lazy evaluation in polars: `df.lazy().filter(...).select(...).\ncollect()`. Polars optimizes the query plan, pushing filters and\nprojections down. In pandas, avoid `apply()` with Python functions —\nuse vectorized operations. Use categorical dtype for low-cardinality\nstrings. Profile with `%timeit` and check memory with\n`df.estimated_size()` (polars).\n\n## Gotchas\n\nAgent-specific failure modes — provider-neutral pause-and-self-check items:\n\n- **Using `apply()` with a Python function in pandas for performance-sensitive code.** `apply()` drops to a Python loop, which is 10–100x slower than vectorized operations. Use built-in string methods, `map()`, `np.where()`, or Polars expressions instead.\n- **Joining without checking for duplicates first.** A join on a key that has duplicates in either table causes a row explosion (M × N rows for M and N matching rows). Check `df[\"key\"].is_unique` before joining; validate with `validate=\"m:1\"` in pandas.\n- **Auto-detecting date formats or dtypes on read.** Pandas' auto-detection of dates and dtypes is unreliable across file versions and locales. Always specify `dtype=` and `parse_dates=` (pandas) or `schema=` (polars) explicitly on read. A column that auto-parsed as `datetime64` in development may parse as `object` on a slightly different CSV in production.\n- **Ignoring naive datetimes and timezone bugs.** A naive datetime (no timezone) compared to a tz-aware datetime raises an error in polars and silently produces wrong results in some pandas operations. Always be explicit about timezones — parse with an explicit `time_zone=` and convert to UTC at the boundary.\n- **Using polars with pandas `apply()` mental model.** Polars expressions are lazy and operate on the whole column — there is no row-by-row loop. Translating pandas `apply()` to polars `map_elements()` (element-wise Python) negates polars' performance advantage. Rewrite in terms of polars expression API.\n- **Not using lazy evaluation in polars for large datasets.** Calling `pl.read_csv(...)` eagerly loads the entire file into memory. Use `pl.scan_csv(...).lazy()` and chain `.filter()`, `.select()`, `.collect()` so polars can push projections and filters down and avoid materializing intermediate results.\n- **Storing multiple logical tables in one DataFrame.** Mixing two different entities (orders and line items) in one DataFrame with repeated parent-level columns produces confusing GroupBy results and joins. Separate into clean normalized DataFrames and join when needed.\n\n## Full reference\n\n### API comparison — reading data\n\n```python\n# Pandas\ndf = pd.read_csv(\"data.csv\", usecols=[\"a\", \"b\"], dtype={\"a\": \"int64\"})\ndf = pd.read_parquet(\"data.parquet\", columns=[\"a\", \"b\"])\n\n# Polars\ndf = pl.read_csv(\"data.csv\", columns=[\"a\", \"b\"], schema={\"a\": pl.Int64})\ndf = pl.read_parquet(\"data.parquet\", columns=[\"a\", \"b\"])\n\n# Polars lazy (recommended for large files)\nlf = pl.scan_csv(\"data.csv\")\ndf = lf.filter(...).select(...).collect()\n```\n\n### Selecting columns\n\n```python\n# Pandas\ndf[[\"name\", \"age\"]]\ndf.loc[:, \"name\":\"age\"]\n\n# Polars\ndf.select(\"name\", \"age\")\ndf.select(pl.col(\"name\"), pl.col(\"age\"))\ndf.select(pl.col(\"^sales_.*$\"))  # regex column selection\n```\n\n### Filtering rows\n\n```python\n# Pandas\ndf[df[\"age\"] > 30]\ndf[(df[\"age\"] > 30) & (df[\"city\"] == \"Berlin\")]\ndf.query(\"age > 30 and city == 'Berlin'\")\n\n# Polars\ndf.filter(pl.col(\"age\") > 30)\ndf.filter((pl.col(\"age\") > 30) & (pl.col(\"city\") == \"Berlin\"))\n```\n\n### Adding \u002F transforming columns\n\n```python\n# Pandas\ndf[\"total\"] = df[\"price\"] * df[\"qty\"]\ndf = df.assign(total=lambda d: d[\"price\"] * d[\"qty\"])\n\n# Polars\ndf = df.with_columns((pl.col(\"price\") * pl.col(\"qty\")).alias(\"total\"))\ndf = df.with_columns(\n    pl.col(\"name\").str.to_uppercase().alias(\"name_upper\"),\n    (pl.col(\"price\") * 1.1).alias(\"price_with_tax\"),\n)\n```\n\n### GroupBy and aggregation\n\n```python\n# Pandas\ndf.groupby(\"category\").agg(\n    avg_price=(\"price\", \"mean\"),\n    total_qty=(\"qty\", \"sum\"),\n    count=(\"id\", \"count\"),\n).reset_index()\n\n# Polars\ndf.group_by(\"category\").agg(\n    pl.col(\"price\").mean().alias(\"avg_price\"),\n    pl.col(\"qty\").sum().alias(\"total_qty\"),\n    pl.col(\"id\").count().alias(\"count\"),\n)\n```\n\n### Window functions\n\n```python\n# Pandas\ndf[\"rank\"] = df.groupby(\"category\")[\"price\"].rank(ascending=False)\ndf[\"avg_in_group\"] = df.groupby(\"category\")[\"price\"].transform(\"mean\")\n\n# Polars\ndf = df.with_columns(\n    pl.col(\"price\").rank(descending=True).over(\"category\").alias(\"rank\"),\n    pl.col(\"price\").mean().over(\"category\").alias(\"avg_in_group\"),\n)\n```\n\n### Joins\n\n```python\n# Pandas\npd.merge(left, right, on=\"id\", how=\"left\")\npd.merge(left, right, left_on=\"user_id\", right_on=\"id\", how=\"inner\")\n\n# Polars\nleft.join(right, on=\"id\", how=\"left\")\nleft.join(right, left_on=\"user_id\", right_on=\"id\", how=\"inner\")\n\n# Anti-join (rows in left NOT in right)\nleft[~left[\"id\"].isin(right[\"id\"])]            # pandas\nleft.join(right, on=\"id\", how=\"anti\")          # polars\n```\n\n### Pivot and melt\n\n```python\n# Pivot — Pandas\ndf.pivot_table(index=\"date\", columns=\"product\", values=\"sales\", aggfunc=\"sum\")\n# Pivot — Polars\ndf.pivot(on=\"product\", index=\"date\", values=\"sales\", aggregate_function=\"sum\")\n\n# Melt — Pandas\ndf.melt(id_vars=[\"date\"], value_vars=[\"product_a\", \"product_b\"],\n        var_name=\"product\", value_name=\"sales\")\n# Melt — Polars\ndf.unpivot(index=\"date\", on=[\"product_a\", \"product_b\"],\n           variable_name=\"product\", value_name=\"sales\")\n```\n\n### Missing data\n\n```python\n# Pandas\ndf.isna().sum()\ndf.fillna({\"price\": 0, \"name\": \"Unknown\"})\ndf.dropna(subset=[\"critical_col\"])\ndf[\"value\"].ffill()\n\n# Polars\ndf.null_count()\ndf.with_columns(\n    pl.col(\"price\").fill_null(0),\n    pl.col(\"name\").fill_null(\"Unknown\"),\n)\ndf.drop_nulls(subset=[\"critical_col\"])\ndf.with_columns(pl.col(\"value\").forward_fill())\n```\n\n### When to use which\n\n| Scenario | Recommendation |\n|---|---|\n| Exploratory data analysis | Pandas (Jupyter, wider ecosystem) |\n| Dataset > 1 GB | Polars (lower memory, parallel) |\n| Production data pipeline | Polars (lazy eval, deterministic) |\n| sklearn \u002F statsmodels integration | Pandas (native support) |\n| Streaming \u002F chunked processing | Polars lazy or pandas chunks |\n| Simple one-off scripts | Either — use what the team knows |\n| Multi-threaded workloads | Polars (releases GIL, built-in parallelism) |\n\n### Worked examples\n\n- **Large CSV → filter → group stats:** Polars lazy mode, scan the\n  CSV, filter before collect, group with multiple aggregations in\n  one `.agg()`, sort, write Parquet.\n- **Pandas → polars conversion:** Replace `.loc[]` with\n  `filter()`\u002F`select()`, convert `groupby().agg()` to polars\n  `group_by().agg()` with `pl.col()` expressions, drop index-based\n  ops, add `.lazy()`\u002F`.collect()` for optimization.\n- **Messy dataset cleanup:** Drop exact duplicates, identify\n  high-null columns, fill or drop based on context, lowercase and\n  strip strings, parse dates with explicit format, validate value\n  ranges via filter expressions.\n\n### References\n\n- [Pandas Documentation](https:\u002F\u002Fpandas.pydata.org\u002Fdocs\u002F)\n- [Polars User Guide](https:\u002F\u002Fdocs.pola.rs\u002F)\n",{"name":5,"metadata":49,"description":56},{"processkit":50},{"id":51,"created":52,"version":53,"category":54,"apiVersion":55},"SKILL-pandas-polars",1775433600000,"1.1.0","data-ai","processkit.projectious.work\u002Fv1","DataFrame operations with pandas and polars — groupby, joins, reshaping, performance. Use when manipulating tabular data, choosing between pandas and polars, optimizing DataFrame code, or translating between the two libraries.\n","main","4876a92620ee792aac2ac9a1a806322d8a572367","archive\u002Fprocesskit-install-20261101\u002Fcontext\u002Ftemplates\u002Fprocesskit\u002Fv0.22.0\u002Fcontext\u002Fskills\u002Fdata-ai\u002Fpandas-polars","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit","https:\u002F\u002Fprojectious-work.github.io\u002Fprocesskit\u002F","https:\u002F\u002Fgithub.com\u002Fprojectious-work","2026-11-11 23:24:18",{"source":8,"skillName":5,"commands":65,"prompt":75,"zipAvailable":76},[66,69,72],{"tool":67,"command":68},"npx","npx skills add projectious-work\u002Fprocesskit --skill pandas-polars",{"tool":70,"command":71},"bunx","bunx skills add projectious-work\u002Fprocesskit --skill pandas-polars",{"tool":73,"command":74},"pnpm","pnpm dlx skills add projectious-work\u002Fprocesskit --skill pandas-polars","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Farchive\u002Fprocesskit-install-20261101\u002Fcontext\u002Ftemplates\u002Fprocesskit\u002Fv0.22.0\u002Fcontext\u002Fskills\u002Fdata-ai\u002Fpandas-polars\u002FSKILL.md",true,[78],{"path":79,"resourceType":80,"resourceTypeDesc":81,"sizeBytes":82,"sourceHtmlUrl":83},"references\u002Fapi-comparison.md","REFERENCE","参考文档",4106,"https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Farchive\u002Fprocesskit-install-20261101\u002Fcontext\u002Ftemplates\u002Fprocesskit\u002Fv0.22.0\u002Fcontext\u002Fskills\u002Fdata-ai\u002Fpandas-polars\u002Freferences\u002Fapi-comparison.md",[85,109,129,150,167,185],{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":94,"publishedAt":44},"01M6X5MTC89B4ZGXW3JVSQCPF8","skill-management","Manage Skill package manifests as v1 ontology entities.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Fsrc\u002Fcontext\u002Fskills\u002Fprocesskit\u002Fskill-management\u002FSKILL.md",[91],{"categoryNo":92,"categoryName":93,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[95,96,97,98,99,100,37,101,102,103,87,104,105,106,41,107,108],"management","skill management","create","update","package","alpha","list","manage","ontology","get","as","manifests","skill package","transition",{"skillNo":110,"name":111,"nameZh":6,"description":112,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":113,"categories":114,"keywords":116,"publishedAt":44},"01M6X5MCV74NHW5MEDNHZRF2C8","schema-management","Manage generated processkit schema contracts through deterministic MCP rebuild and inspection workflows.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Fsrc\u002Fcontext\u002Fskills\u002Fprocesskit\u002Fschema-management\u002FSKILL.md",[115],{"categoryNo":92,"categoryName":93,"taxonomyType":21,"taxonomyTypeDesc":22},[95,117,118,119,120,121,100,37,122,123,102,124,125,126,104,127,128],"schema","schema management","MCP","workflows","validation","generated","rebuild","contracts","wed","deterministic","contract","mode",{"skillNo":130,"name":131,"nameZh":6,"description":132,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":133,"categories":134,"keywords":135,"publishedAt":44},"01M6X5M5G6YJ96AMZGSCZJ1D13","proposition-management","Record claims, risks, beliefs, world facts, WSJF estimates, and assumptions through the Proposition interface.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Fsrc\u002Fcontext\u002Fskills\u002Fprocesskit\u002Fproposition-management\u002FSKILL.md",[],[95,136,137,97,98,138,100,37,139,140,141,142,104,143,144,145,146,147,148,149],"proposition","proposition management","index","log","world","record","query","creation","event","interface","updates","facts","claims","WSJF",{"skillNo":151,"name":152,"nameZh":6,"description":153,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":154,"categories":155,"keywords":156,"publishedAt":44},"01M6X5KMQ83QN9M6MPWZM7344F","ontology-management","Manage generated v1 ontology entities that do not require a dedicated domain server.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Fsrc\u002Fcontext\u002Fskills\u002Fprocesskit\u002Fontology-management\u002FSKILL.md",[],[95,103,157,97,100,158,159,122,160,139,102,161,162,104,163,164,144,165,41,166],"ontology management","lifecycle","archive","roadmap","do","domain","iteration","keep","measurement","generic",{"skillNo":168,"name":169,"nameZh":6,"description":170,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":171,"categories":172,"keywords":173,"publishedAt":44},"01M6X5KCY2DN3SM21RERSFNEBT","okf-compatibility","Import, export, and validate conformant Open Knowledge Format bundles.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Fsrc\u002Fcontext\u002Fskills\u002Fprocesskit\u002Fokf-compatibility\u002FSKILL.md",[],[174,175,176,177,100,178,179,180,37,181,182,183,184],"okf","compatibility","okf compatibility","validate","open","knowledge","format","export","bundles","bundle","import",{"skillNo":186,"name":187,"nameZh":6,"description":188,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":189,"categories":190,"keywords":191,"publishedAt":44},"01M6X5JA8S5PNGJVFW4EQY8T83","capability-management","Manage versioned Capability entities and their governed lifecycle.","https:\u002F\u002Fgithub.com\u002Fprojectious-work\u002Fprocesskit\u002Fblob\u002Fmain\u002Fsrc\u002Fcontext\u002Fskills\u002Fprocesskit\u002Fcapability-management\u002FSKILL.md",[],[95,192,193,97,98,138,100,158,37,101,139,194,102,141,104,164,144,145,195,108],"capability","capability management","queries","changes"]