[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M6HXMAGE9ZAE2WTMSFJYJB6J":3,"skill-same:01M6HXMAGE9ZAE2WTMSFJYJB6J":71},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":23,"publishedAt":38,"linkOnly":39,"markdownContent":40,"markdownBody":41,"frontMatter":42,"refName":43,"commitSha":44,"skillDirectory":45,"forksCount":46,"repositoryHtmlUrl":47,"repositoryHomepageUrl":6,"ownerHtmlUrl":48,"ownerBlogUrl":49,"ownerDescription":6,"updatedAt":50,"install":51,"resources":64},"01M6HXMAGE9ZAE2WTMSFJYJB6J","irregular-excel-parsing",null,"Handle Excel files with irregular headers, merged cells, and unknown header row positions using pattern-matching and index-based extraction.","HKUDS\u002FAgentPilot","HKUDS","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F118165258?v=4","01M6HW8VRNZP64G0P4MVJW2KKQ","✨Data Intelligence Lab@HKU✨","01M6HW8VWZVHBRHYH3T330PFWV",7993,"MIT","https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot\u002Fblob\u002Fmain\u002Fopenspace\u002Flegacy\u002Fv1\u002Fgdpval_bench\u002Fskills\u002Firregular-excel-parsing\u002FSKILL.md",[18],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000004","电子表格","SCENARIO","使用场景",[24,25,26,27,28,29,30,31,32,33,34,35,36,37],"xlsx","parsing","excel parsing","irregular excel parsing","index","extraction","pattern-matching","files","pattern","positions","matching","with","headers","unknown","2026-11-10 10:12:30",false,"---\nname: irregular-excel-parsing\ndescription: Handle Excel files with irregular headers, merged cells, and unknown header row positions using pattern-matching and index-based extraction.\n---\n\n# Irregular Excel File Parsing\n\nUse this skill when you encounter Excel files where standard `pandas.read_excel()` fails due to:\n- Headers not in row 0 (6-9+ header rows common)\n- Merged cells in header area\n- Unknown or inconsistent header row positions\n- Multiple title\u002Fmetadata rows before actual data\n\n## Step-by-Step Instructions\n\n### Step 1: Read Excel Without Headers\n\nFirst, read the entire sheet with `header=None` to get raw data:\n\n```python\nimport pandas as pd\n\n# Read all data without assuming header position\ndf_raw = pd.read_excel('filename.xlsx', sheet_name='Sheet1', header=None)\n```\n\n### Step 2: Scan Rows to Find Header Pattern\n\nSearch for the actual header row by looking for distinctive patterns:\n\n```python\ndef find_header_row(df):\n    \"\"\"Find header row by pattern-matching common column identifiers.\"\"\"\n    header_patterns = [\n        r'Store ID',\n        r'ID\\d{4}',  # ID followed by 4 digits\n        r'Week \\d+',\n        r'Date',\n        r'Store',\n        r'Product',\n        r'ID'\n    ]\n    \n    for row_idx in range(len(df)):\n        row_values = df.iloc[row_idx].astype(str).str.lower()\n        for pattern in header_patterns:\n            if row_values.str.contains(pattern, case=False, regex=True).any():\n                return row_idx\n    \n    # Fallback: return first non-empty row\n    for row_idx in range(len(df)):\n        if df.iloc[row_idx].notna().sum() > 0:\n            return row_idx\n    \n    return 0\n\nheader_row = find_header_row(df_raw)\n```\n\n### Step 3: Extract and Clean Headers\n\nExtract the header row and clean column names:\n\n```python\n# Extract header row\nheaders = df_raw.iloc[header_row].tolist()\n\n# Clean headers: convert to string, strip whitespace, handle NaN\nclean_headers = []\nfor h in headers:\n    if pd.isna(h) or str(h).strip() == '':\n        clean_headers.append(f'col_{len(clean_headers)}')\n    else:\n        clean_headers.append(str(h).strip())\n\n# Handle duplicate headers by adding suffix\nfrom collections import Counter\nheader_counts = Counter(clean_headers)\nfinal_headers = []\nfor h in clean_headers:\n    if header_counts[h] > 1:\n        final_headers.append(f\"{h}_{header_counts[h]}\")\n        header_counts[h] -= 1\n    else:\n        final_headers.append(h)\n```\n\n### Step 4: Extract Data Rows\n\nExtract data starting from the row after headers:\n\n```python\n# Get data rows (everything after header)\ndata_df = df_raw.iloc[header_row + 1:].copy()\ndata_df.columns = final_headers\n\n# Reset index\ndata_df = data_df.reset_index(drop=True)\n\n# Remove completely empty rows\ndata_df = data_df.dropna(how='all')\n```\n\n### Step 5: Validate and Clean Data\n\nPerform basic validation and type conversion:\n\n```python\n# Identify ID columns and preserve as string\nfor col in data_df.columns:\n    if 'id' in col.lower() or 'code' in col.lower():\n        data_df[col] = data_df[col].astype(str).str.strip()\n\n# Convert numeric columns\nnumeric_cols = data_df.select_dtypes(include=['float64', 'int64']).columns\nfor col in numeric_cols:\n    data_df[col] = pd.to_numeric(data_df[col], errors='coerce')\n\n# Remove rows with invalid critical data\nif 'Store ID' in data_df.columns:\n    data_df = data_df[data_df['Store ID'].notna() & (data_df['Store ID'] != '')]\n```\n\n## Complete Example Function\n\n```python\ndef parse_irregular_excel(filepath, sheet_name=0):\n    \"\"\"Parse Excel file with unknown\u002Firregular header structure.\"\"\"\n    import pandas as pd\n    import re\n    from collections import Counter\n    \n    # Step 1: Read raw\n    df_raw = pd.read_excel(filepath, sheet_name=sheet_name, header=None)\n    \n    # Step 2: Find header row\n    patterns = [r'Store ID', r'ID\\d{4}', r'Week', r'Date', r'Store']\n    header_row = 0\n    for idx in range(min(15, len(df_raw))):  # Check first 15 rows\n        row_str = ' '.join(df_raw.iloc[idx].astype(str))\n        for pattern in patterns:\n            if re.search(pattern, row_str, re.IGNORECASE):\n                header_row = idx\n                break\n    \n    # Step 3: Extract headers\n    headers = [str(h).strip() if pd.notna(h) else f'col_{i}' \n               for i, h in enumerate(df_raw.iloc[header_row])]\n    \n    # Handle duplicates\n    counts = Counter(headers)\n    final_headers = []\n    for h in headers:\n        if counts[h] > 1:\n            final_headers.append(f\"{h}_{counts[h]}\")\n            counts[h] -= 1\n        else:\n            final_headers.append(h)\n    \n    # Step 4: Extract data\n    data_df = df_raw.iloc[header_row + 1:].copy()\n    data_df.columns = final_headers\n    data_df = data_df.dropna(how='all').reset_index(drop=True)\n    \n    return data_df\n```\n\n## When to Use This Pattern\n\n- ✅ Excel files with 6-9+ title\u002Fmetadata rows before data\n- ✅ Merged cells in header area causing misalignment\n- ✅ Headers not in predictable positions\n- ✅ Multiple sheets with inconsistent structures\n\n## When NOT to Use\n\n- ❌ Standard Excel files with headers in row 0 (use `read_excel()` directly)\n- ❌ Files with consistent, known structure (use explicit `header=` parameter)\n- ❌ When you know exact header row position (specify it directly)\n\n## Tips\n\n1. **Always inspect first**: Use `df_raw.head(20)` to visualize structure before parsing\n2. **Pattern flexibility**: Adjust regex patterns based on your specific column naming conventions\n3. **Handle merged cells**: Merged cells often result in NaN values - fill strategically if needed\n4. **Save for reuse**: Once you determine the correct header row for a file type, cache this for future runs","# Irregular Excel File Parsing\n\nUse this skill when you encounter Excel files where standard `pandas.read_excel()` fails due to:\n- Headers not in row 0 (6-9+ header rows common)\n- Merged cells in header area\n- Unknown or inconsistent header row positions\n- Multiple title\u002Fmetadata rows before actual data\n\n## Step-by-Step Instructions\n\n### Step 1: Read Excel Without Headers\n\nFirst, read the entire sheet with `header=None` to get raw data:\n\n```python\nimport pandas as pd\n\n# Read all data without assuming header position\ndf_raw = pd.read_excel('filename.xlsx', sheet_name='Sheet1', header=None)\n```\n\n### Step 2: Scan Rows to Find Header Pattern\n\nSearch for the actual header row by looking for distinctive patterns:\n\n```python\ndef find_header_row(df):\n    \"\"\"Find header row by pattern-matching common column identifiers.\"\"\"\n    header_patterns = [\n        r'Store ID',\n        r'ID\\d{4}',  # ID followed by 4 digits\n        r'Week \\d+',\n        r'Date',\n        r'Store',\n        r'Product',\n        r'ID'\n    ]\n    \n    for row_idx in range(len(df)):\n        row_values = df.iloc[row_idx].astype(str).str.lower()\n        for pattern in header_patterns:\n            if row_values.str.contains(pattern, case=False, regex=True).any():\n                return row_idx\n    \n    # Fallback: return first non-empty row\n    for row_idx in range(len(df)):\n        if df.iloc[row_idx].notna().sum() > 0:\n            return row_idx\n    \n    return 0\n\nheader_row = find_header_row(df_raw)\n```\n\n### Step 3: Extract and Clean Headers\n\nExtract the header row and clean column names:\n\n```python\n# Extract header row\nheaders = df_raw.iloc[header_row].tolist()\n\n# Clean headers: convert to string, strip whitespace, handle NaN\nclean_headers = []\nfor h in headers:\n    if pd.isna(h) or str(h).strip() == '':\n        clean_headers.append(f'col_{len(clean_headers)}')\n    else:\n        clean_headers.append(str(h).strip())\n\n# Handle duplicate headers by adding suffix\nfrom collections import Counter\nheader_counts = Counter(clean_headers)\nfinal_headers = []\nfor h in clean_headers:\n    if header_counts[h] > 1:\n        final_headers.append(f\"{h}_{header_counts[h]}\")\n        header_counts[h] -= 1\n    else:\n        final_headers.append(h)\n```\n\n### Step 4: Extract Data Rows\n\nExtract data starting from the row after headers:\n\n```python\n# Get data rows (everything after header)\ndata_df = df_raw.iloc[header_row + 1:].copy()\ndata_df.columns = final_headers\n\n# Reset index\ndata_df = data_df.reset_index(drop=True)\n\n# Remove completely empty rows\ndata_df = data_df.dropna(how='all')\n```\n\n### Step 5: Validate and Clean Data\n\nPerform basic validation and type conversion:\n\n```python\n# Identify ID columns and preserve as string\nfor col in data_df.columns:\n    if 'id' in col.lower() or 'code' in col.lower():\n        data_df[col] = data_df[col].astype(str).str.strip()\n\n# Convert numeric columns\nnumeric_cols = data_df.select_dtypes(include=['float64', 'int64']).columns\nfor col in numeric_cols:\n    data_df[col] = pd.to_numeric(data_df[col], errors='coerce')\n\n# Remove rows with invalid critical data\nif 'Store ID' in data_df.columns:\n    data_df = data_df[data_df['Store ID'].notna() & (data_df['Store ID'] != '')]\n```\n\n## Complete Example Function\n\n```python\ndef parse_irregular_excel(filepath, sheet_name=0):\n    \"\"\"Parse Excel file with unknown\u002Firregular header structure.\"\"\"\n    import pandas as pd\n    import re\n    from collections import Counter\n    \n    # Step 1: Read raw\n    df_raw = pd.read_excel(filepath, sheet_name=sheet_name, header=None)\n    \n    # Step 2: Find header row\n    patterns = [r'Store ID', r'ID\\d{4}', r'Week', r'Date', r'Store']\n    header_row = 0\n    for idx in range(min(15, len(df_raw))):  # Check first 15 rows\n        row_str = ' '.join(df_raw.iloc[idx].astype(str))\n        for pattern in patterns:\n            if re.search(pattern, row_str, re.IGNORECASE):\n                header_row = idx\n                break\n    \n    # Step 3: Extract headers\n    headers = [str(h).strip() if pd.notna(h) else f'col_{i}' \n               for i, h in enumerate(df_raw.iloc[header_row])]\n    \n    # Handle duplicates\n    counts = Counter(headers)\n    final_headers = []\n    for h in headers:\n        if counts[h] > 1:\n            final_headers.append(f\"{h}_{counts[h]}\")\n            counts[h] -= 1\n        else:\n            final_headers.append(h)\n    \n    # Step 4: Extract data\n    data_df = df_raw.iloc[header_row + 1:].copy()\n    data_df.columns = final_headers\n    data_df = data_df.dropna(how='all').reset_index(drop=True)\n    \n    return data_df\n```\n\n## When to Use This Pattern\n\n- ✅ Excel files with 6-9+ title\u002Fmetadata rows before data\n- ✅ Merged cells in header area causing misalignment\n- ✅ Headers not in predictable positions\n- ✅ Multiple sheets with inconsistent structures\n\n## When NOT to Use\n\n- ❌ Standard Excel files with headers in row 0 (use `read_excel()` directly)\n- ❌ Files with consistent, known structure (use explicit `header=` parameter)\n- ❌ When you know exact header row position (specify it directly)\n\n## Tips\n\n1. **Always inspect first**: Use `df_raw.head(20)` to visualize structure before parsing\n2. **Pattern flexibility**: Adjust regex patterns based on your specific column naming conventions\n3. **Handle merged cells**: Merged cells often result in NaN values - fill strategically if needed\n4. **Save for reuse**: Once you determine the correct header row for a file type, cache this for future runs",{"name":5,"description":7},"main","6b84b0689ca0d9e7b1e4061188ff6b60758e0f8a","openspace\u002Flegacy\u002Fv1\u002Fgdpval_bench\u002Fskills\u002Firregular-excel-parsing",940,"https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot","https:\u002F\u002Fgithub.com\u002FHKUDS","https:\u002F\u002Fsites.google.com\u002Fview\u002Fchaoh","2026-11-06 23:04:01",{"source":8,"skillName":5,"commands":52,"prompt":62,"zipAvailable":63},[53,56,59],{"tool":54,"command":55},"npx","npx skills add HKUDS\u002FAgentPilot --skill irregular-excel-parsing",{"tool":57,"command":58},"bunx","bunx skills add HKUDS\u002FAgentPilot --skill irregular-excel-parsing",{"tool":60,"command":61},"pnpm","pnpm dlx skills add HKUDS\u002FAgentPilot --skill irregular-excel-parsing","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot\u002Fblob\u002Fmain\u002Fopenspace\u002Flegacy\u002Fv1\u002Fgdpval_bench\u002Fskills\u002Firregular-excel-parsing\u002FSKILL.md",true,[65],{"path":66,"resourceType":67,"resourceTypeDesc":68,"sizeBytes":69,"sourceHtmlUrl":70},".skill_id","OTHER","其他",37,"https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot\u002Fblob\u002Fmain\u002Fopenspace\u002Flegacy\u002Fv1\u002Fgdpval_bench\u002Fskills\u002Firregular-excel-parsing\u002F.skill_id",[72,91,110,133,157,179],{"skillNo":73,"name":74,"nameZh":6,"description":75,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":76,"categories":77,"keywords":78,"publishedAt":38},"01M6HYB297NDABRERJRVG0K8R0","agentpilot","Set up AgentPilot and use its single entry to prepare missing inputs and run configured RSI experiments. Use when the user requests AgentPilot installation or work with AgentPilot.","https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot\u002Fblob\u002Fmain\u002Fplugins\u002Fagentpilot\u002Fskills\u002Fagentpilot\u002FSKILL.md",[],[74,79,80,35,81,82,83,84,85,86,87,88,89,90],"prepare","requests","missing","installation","rsi","up","use","user","work","SET","run","experiments",{"skillNo":92,"name":93,"nameZh":6,"description":94,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":95,"categories":96,"keywords":97,"publishedAt":38},"01M6HYAYWXTTE1J7DJM0MFNQ4H","remember","Review auto-memory entries and propose promotions to OPENSPACE.md, OPENSPACE.local.md, or retained auto-memory topics; detect duplicates, stale facts, conflicts, and ambiguous entries before making changes.","https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot\u002Fblob\u002Fmain\u002Fopenspace\u002Fopenspace\u002Fskills\u002Fremember\u002FSKILL.md",[],[93,98,99,100,101,102,103,104,105,106,107,108,109],"memory","review","auto","openspace","changes","facts","detect","entries","local","conflicts","stale","duplicates",{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":114,"categories":115,"keywords":119,"publishedAt":38},"01M6HYAVMPM4SBDV60R2B5HPV8","skill-discovery","Search reusable OpenSpace skills locally, and browse cloud packages step by step when a cloud skill may help. Reusing proven skills saves tokens, improves reliability, and extends your capabilities beyond built-in tools.","https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot\u002Fblob\u002Fmain\u002Fopenspace\u002Fopenspace\u002Fhost_skills\u002Fskill-discovery\u002FSKILL.md",[116],{"categoryNo":117,"categoryName":118,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000023","Skill与智能体开发",[120,121,122,101,123,124,125,126,127,128,129,130,131,132],"skill discovery","discovery","reusable","step by step","tokens","packages","help","cloud","reliability","capabilities","browse","search","Tools",{"skillNo":134,"name":135,"nameZh":6,"description":136,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":137,"categories":138,"keywords":140,"publishedAt":38},"01M6HYAR811B3RB0M50RBEDGN6","delegate-task","Delegate tasks to OpenSpace — a full-stack autonomous worker for coding, DevOps, web research, and desktop automation, backed by an extensive MCP tool and skill library. Skills auto-improve through use, reducing token consumption over time. A cloud community lets agents share and collectively evolve reusable skills.","https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot\u002Fblob\u002Fmain\u002Fopenspace\u002Fopenspace\u002Fhost_skills\u002Fdelegate-task\u002FSKILL.md",[139],{"categoryNo":117,"categoryName":118,"taxonomyType":21,"taxonomyTypeDesc":22},[141,142,143,101,144,145,146,147,148,122,100,85,149,150,151,152,153,154,155,156],"delegate","task","delegate task","time","improve","desktop automation","worker","community","autonomous","tool","stack","for","share","tasks","automation","MCP",{"skillNo":158,"name":159,"nameZh":6,"description":160,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":161,"categories":162,"keywords":163,"publishedAt":38},"01M6HYAM9FSGV6E52AYENDQXYH","write-special-chars-ts-check","Workaround for write_file failures caused by special characters (apostrophes, backticks, template literals) using a Python heredoc script, plus a reliable TypeScript type-checking step that filters output to only the newly-written files to avoid being misled by pre-existing errors.","https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot\u002Fblob\u002Fmain\u002Fopenspace\u002Flegacy\u002Fv1\u002Fshowcase\u002Fskills\u002Fwrite-special-chars-ts-check\u002FSKILL.md",[],[164,165,166,167,31,168,169,170,171,172,173,174,175,176,152,177,178],"write special chars ts check","write","check","type","filters","only","template","file","python","typescript","output","Existing","type-checking","errors","script",{"skillNo":180,"name":181,"nameZh":6,"description":182,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":183,"categories":184,"keywords":185,"publishedAt":38},"01M6HYAH05HGV7SVWSVR8PJRQD","write-special-chars-readback","Workaround for write_file failures caused by special characters (apostrophes, backticks, template literals) using a Python heredoc script, with mandatory post-write verification via read_file and cleanup of \u002Ftmp helper scripts.","https:\u002F\u002Fgithub.com\u002FHKUDS\u002FAgentPilot\u002Fblob\u002Fmain\u002Fopenspace\u002Flegacy\u002Fv1\u002Fshowcase\u002Fskills\u002Fwrite-special-chars-readback\u002FSKILL.md",[],[186,187,165,35,152,178,188,189,190,191,192,170,171,193,194,172],"write special chars readback","readback","helper","post","via","scripts","read","verification","cleanup"]