[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7BV0CESDJAFBZBQRCBVBMCQ":3,"skill-same:01M7BV0CESDJAFBZBQRCBVBMCQ":160},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":85,"commitSha":86,"skillDirectory":87,"forksCount":88,"repositoryHtmlUrl":89,"repositoryHomepageUrl":6,"ownerHtmlUrl":90,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":91,"install":92,"resources":104},"01M7BV0CESDJAFBZBQRCBVBMCQ","senior-data-engineer",null,"Implement and debug a Flink DataStream job that sessionizes task SUBMIT events per job (10-minute inactivity gap), computes the maximum tasks in any stage, and emits results once the job is finished.","linyouxue\u002Fskill-repair-benchmark","linyouxue","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F216151464?v=4","01M7BJEVCXKD74KH6FG9CBVSQB","林宥学","01M7BJEVE02NP217V93BKZFC0J",0,"MIT","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: senior-data-engineer\ntitle: Senior Data Engineer Skill Package\ndescription: Implement and debug a Flink DataStream job that sessionizes task SUBMIT\n  events per job (10-minute inactivity gap), computes the maximum tasks in any stage,\n  and emits results once the job is finished.\ndomain: engineering\nsubdomain: data-engineering\ndifficulty: advanced\ntime-saved: 'TODO: Quantify time savings'\nfrequency: 'TODO: Estimate usage frequency'\nuse-cases:\n- Designing data pipelines for ETL\u002FELT processes\n- Building data warehouses and data lakes\n- Implementing data quality and governance frameworks\n- Creating analytics dashboards and reporting\n- Building real-time streaming pipelines with Kafka and Flink\n- Implementing exactly-once streaming semantics\n- Monitoring streaming quality (consumer lag, data freshness, schema drift)\nrelated-agents: []\nrelated-skills: []\nrelated-commands: []\norchestrated-by: []\ndependencies:\n  scripts: []\n  references: []\n  assets: []\ncompatibility: 'Python 3.8+; platforms: macos, linux, windows'\ntech-stack:\n- Python\n- SQL\n- Apache Spark\n- Airflow\n- dbt\n- Apache Kafka\n- Apache Flink\n- AWS Kinesis\n- Spark Structured Streaming\n- Kafka Streams\n- PostgreSQL\n- BigQuery\n- Snowflake\n- Docker\n- Schema Registry\nexamples:\n- title: Example Usage\n  input: 'TODO: Add example input for senior-data-engineer'\n  output: 'TODO: Add expected output'\nstats:\n  downloads: 0\n  stars: 0\n  rating: 0.0\n  reviews: 0\nversion: v2.0.0\nauthor: Claude Skills Team\ncontributors: []\ncreated: 2025-10-20\nupdated: 2025-12-16\nlicense: MIT\ntags:\n- architecture\n- data\n- design\n- engineer\n- engineering\n- senior\n- streaming\n- kafka\n- flink\n- real-time\nfeatured: false\nverified: true\n---\n\n## Steps\n1. Read schema docs (`format.pdf`, `ClusterData2011_2.md`) and implement the required datatypes (e.g., `TaskEvent`, `JobEvent`) under `clusterdata.datatypes` matching the CSV column order and microsecond timestamps.\n2. In the Flink job:\n   - Ingest a single gzipped CSV shard for task events and job events.\n   - Parse events and assign event-time timestamps in **microseconds**; use a watermark strategy appropriate for the input (bounded files).\n3. Filter and stage task submissions:\n   - Keep only task `SUBMIT` events.\n   - `keyBy(jobId)` and build **event-time session “stages”** with a 10-minute inactivity gap (`10 * 60 * 1_000_000` microseconds).\n   - Count the number of SUBMIT events per session window.\n   - Reduce per `jobId` to the **maximum** session count (longest stage).\n4. Emit only when the job is done:\n   - From job events, derive a “terminal” signal per `jobId` (include at least `FINISH`, and also treat `KILL` as terminal when present).\n   - Join \u002F connect the terminal signal with the computed max-stage count so output is produced “once the job has finished” (jobs with no matching submits in the chosen task shard may legitimately produce `0`).\n5. Prevent Flink generic type-erasure runtime failures:\n   - If using generic `SourceFunction` \u002F parsing operators (e.g., `GzipCsvSourceFunction\u003CT>`), **add explicit type hints** on the resulting `DataStream` (e.g., `.returns(TaskEvent.class)` \u002F `.returns(JobEvent.class)`) at the point where Flink can’t infer `T`.\n6. Write results to the provided `output` path:\n   - One tuple per line in exactly: `(jobId,longestStageTaskCount)`\n   - Ordering not required.\n## Expected Result\nThe job compiles and runs on Flink without `InvalidTypesException` from generics, and writes a local output file containing one line per jobId with the longest stage’s SUBMIT-count, emitting results only after terminal job completion (including killed jobs).\n","## Steps\n1. Read schema docs (`format.pdf`, `ClusterData2011_2.md`) and implement the required datatypes (e.g., `TaskEvent`, `JobEvent`) under `clusterdata.datatypes` matching the CSV column order and microsecond timestamps.\n2. In the Flink job:\n   - Ingest a single gzipped CSV shard for task events and job events.\n   - Parse events and assign event-time timestamps in **microseconds**; use a watermark strategy appropriate for the input (bounded files).\n3. Filter and stage task submissions:\n   - Keep only task `SUBMIT` events.\n   - `keyBy(jobId)` and build **event-time session “stages”** with a 10-minute inactivity gap (`10 * 60 * 1_000_000` microseconds).\n   - Count the number of SUBMIT events per session window.\n   - Reduce per `jobId` to the **maximum** session count (longest stage).\n4. Emit only when the job is done:\n   - From job events, derive a “terminal” signal per `jobId` (include at least `FINISH`, and also treat `KILL` as terminal when present).\n   - Join \u002F connect the terminal signal with the computed max-stage count so output is produced “once the job has finished” (jobs with no matching submits in the chosen task shard may legitimately produce `0`).\n5. Prevent Flink generic type-erasure runtime failures:\n   - If using generic `SourceFunction` \u002F parsing operators (e.g., `GzipCsvSourceFunction\u003CT>`), **add explicit type hints** on the resulting `DataStream` (e.g., `.returns(TaskEvent.class)` \u002F `.returns(JobEvent.class)`) at the point where Flink can’t infer `T`.\n6. Write results to the provided `output` path:\n   - One tuple per line in exactly: `(jobId,longestStageTaskCount)`\n   - Ordering not required.\n## Expected Result\nThe job compiles and runs on Flink without `InvalidTypesException` from generics, and writes a local output file containing one line per jobId with the longest stage’s SUBMIT-count, emitting results only after terminal job completion (including killed jobs).\n",{"name":5,"tags":24,"stats":35,"title":36,"author":37,"domain":29,"created":38,"license":15,"updated":39,"version":40,"examples":41,"featured":20,"verified":46,"frequency":47,"subdomain":48,"use-cases":49,"difficulty":57,"tech-stack":58,"time-saved":74,"description":7,"contributors":75,"dependencies":76,"compatibility":80,"related-agents":81,"related-skills":82,"orchestrated-by":83,"related-commands":84},[25,26,27,28,29,30,31,32,33,34],"architecture","data","design","engineer","engineering","senior","streaming","kafka","flink","real-time",{"stars":14,"rating":14,"reviews":14,"downloads":14},"Senior Data Engineer Skill Package","Claude Skills Team",1760918400000,1765843200000,"v2.0.0",[42],{"input":43,"title":44,"output":45},"TODO: Add example input for senior-data-engineer","Example Usage","TODO: Add expected output",true,"TODO: Estimate usage frequency","data-engineering",[50,51,52,53,54,55,56],"Designing data pipelines for ETL\u002FELT processes","Building data warehouses and data lakes","Implementing data quality and governance frameworks","Creating analytics dashboards and reporting","Building real-time streaming pipelines with Kafka and Flink","Implementing exactly-once streaming semantics","Monitoring streaming quality (consumer lag, data freshness, schema drift)","advanced",[59,60,61,62,63,64,65,66,67,68,69,70,71,72,73],"Python","SQL","Apache Spark","Airflow","dbt","Apache Kafka","Apache Flink","AWS Kinesis","Spark Structured Streaming","Kafka Streams","PostgreSQL","BigQuery","Snowflake","Docker","Schema Registry","TODO: Quantify time savings",[],{"assets":77,"scripts":78,"references":79},[],[],[],"Python 3.8+; platforms: macos, linux, windows",[],[],[],[],"main","cdcb6d230f3b15c3c33f8a2bdd16439f43602cbe","evaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer",1,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark","https:\u002F\u002Fgithub.com\u002Flinyouxue","2026-11-14 20:28:35",{"source":8,"skillName":5,"commands":93,"prompt":103,"zipAvailable":46},[94,97,100],{"tool":95,"command":96},"npx","npx skills add linyouxue\u002Fskill-repair-benchmark --skill senior-data-engineer",{"tool":98,"command":99},"bunx","bunx skills add linyouxue\u002Fskill-repair-benchmark --skill senior-data-engineer",{"tool":101,"command":102},"pnpm","pnpm dlx skills add linyouxue\u002Fskill-repair-benchmark --skill senior-data-engineer","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002FSKILL.md",[105,110,113,116,122,126,130,134,140,144,148,152,156],{"path":106,"resourceType":107,"resourceTypeDesc":108,"sizeBytes":14,"sourceHtmlUrl":109},"assets\u002F.gitkeep","ASSET","资产文件","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Fassets\u002F.gitkeep",{"path":111,"resourceType":107,"resourceTypeDesc":108,"sizeBytes":14,"sourceHtmlUrl":112},"assets\u002Fdata-quality-checklist.md","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Fassets\u002Fdata-quality-checklist.md",{"path":114,"resourceType":107,"resourceTypeDesc":108,"sizeBytes":14,"sourceHtmlUrl":115},"assets\u002Fpipeline-design-template.md","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Fassets\u002Fpipeline-design-template.md",{"path":117,"resourceType":118,"resourceTypeDesc":119,"sizeBytes":120,"sourceHtmlUrl":121},"HOW_TO_USE.md","REFERENCE","参考文档",2500,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002FHOW_TO_USE.md",{"path":123,"resourceType":118,"resourceTypeDesc":119,"sizeBytes":124,"sourceHtmlUrl":125},"references\u002Fframeworks.md",42313,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Freferences\u002Fframeworks.md",{"path":127,"resourceType":118,"resourceTypeDesc":119,"sizeBytes":128,"sourceHtmlUrl":129},"references\u002Ftemplates.md",67136,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Freferences\u002Ftemplates.md",{"path":131,"resourceType":118,"resourceTypeDesc":119,"sizeBytes":132,"sourceHtmlUrl":133},"references\u002Ftools.md",39140,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Freferences\u002Ftools.md",{"path":135,"resourceType":136,"resourceTypeDesc":137,"sizeBytes":138,"sourceHtmlUrl":139},"scripts\u002Fdata_quality_validator.py","SCRIPT","脚本",3489,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Fscripts\u002Fdata_quality_validator.py",{"path":141,"resourceType":136,"resourceTypeDesc":137,"sizeBytes":142,"sourceHtmlUrl":143},"scripts\u002Fetl_performance_optimizer.py",3504,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Fscripts\u002Fetl_performance_optimizer.py",{"path":145,"resourceType":136,"resourceTypeDesc":137,"sizeBytes":146,"sourceHtmlUrl":147},"scripts\u002Fkafka_config_generator.py",41177,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Fscripts\u002Fkafka_config_generator.py",{"path":149,"resourceType":136,"resourceTypeDesc":137,"sizeBytes":150,"sourceHtmlUrl":151},"scripts\u002Fpipeline_orchestrator.py",3487,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Fscripts\u002Fpipeline_orchestrator.py",{"path":153,"resourceType":136,"resourceTypeDesc":137,"sizeBytes":154,"sourceHtmlUrl":155},"scripts\u002Fstream_processor.py",60553,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Fscripts\u002Fstream_processor.py",{"path":157,"resourceType":136,"resourceTypeDesc":137,"sizeBytes":158,"sourceHtmlUrl":159},"scripts\u002Fstreaming_quality_validator.py",40005,"https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fevaluation\u002Fresults\u002FMMG2Skill-GPT52-Gold31-20260923\u002Ftasks\u002Fflink-query\u002Frepaired_skill\u002Fsenior-data-engineer\u002Fscripts\u002Fstreaming_quality_validator.py",[161,169,176,183,188,193],{"skillNo":162,"name":163,"nameZh":6,"description":164,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":165,"licenseFileUrl":6,"sourceHtmlUrl":166,"categories":167,"keywords":168,"publishedAt":19},"01M7E1HGVRY73NBJ022CJG13X6","video-frame-extraction","Extract frames from video files and save them as images using OpenCV","Apache-2.0","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fjpg-ocr-stat\u002Fresult\u002Fskilltta-claude4.7\u002Fgroup_03\u002Frun_03\u002Finputs\u002Fskills\u002Fvideo-frame-extraction\u002FSKILL.md",[],[],{"skillNo":170,"name":171,"nameZh":6,"description":172,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":165,"licenseFileUrl":6,"sourceHtmlUrl":173,"categories":174,"keywords":175,"publishedAt":19},"01M7E1HAG5R7CJ9C8NYJS771MB","openai-vision","Analyze images and multi-frame sequences using OpenAI GPT vision models","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fjpg-ocr-stat\u002Fresult\u002Fskilltta-claude4.7\u002Fgroup_03\u002Frun_03\u002Finputs\u002Fskills\u002Fopenai-vision\u002FSKILL.md",[],[],{"skillNo":177,"name":178,"nameZh":6,"description":179,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":165,"licenseFileUrl":6,"sourceHtmlUrl":180,"categories":181,"keywords":182,"publishedAt":19},"01M7E1H7DADKB6QBWKC2KQ63SZ","image-ocr","Extract text content from images using Tesseract OCR via Python, including structured field extraction from receipts and forms","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fjpg-ocr-stat\u002Fresult\u002Fskilltta-claude4.7\u002Fgroup_03\u002Frun_03\u002Finputs\u002Fskills\u002Fimage-ocr\u002FSKILL.md",[],[],{"skillNo":184,"name":163,"nameZh":6,"description":164,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":165,"licenseFileUrl":6,"sourceHtmlUrl":185,"categories":186,"keywords":187,"publishedAt":19},"01M7E1H0WBNT43TENGQY7AVZHD","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fjpg-ocr-stat\u002Fresult\u002Fskilltta-claude4.7\u002Fgroup_03\u002Frun_02\u002Finputs\u002Fskills\u002Fvideo-frame-extraction\u002FSKILL.md",[],[],{"skillNo":189,"name":171,"nameZh":6,"description":172,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":165,"licenseFileUrl":6,"sourceHtmlUrl":190,"categories":191,"keywords":192,"publishedAt":19},"01M7E1GTC7D4N4Q2EF03875J9E","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fjpg-ocr-stat\u002Fresult\u002Fskilltta-claude4.7\u002Fgroup_03\u002Frun_02\u002Finputs\u002Fskills\u002Fopenai-vision\u002FSKILL.md",[],[],{"skillNo":194,"name":178,"nameZh":6,"description":179,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":165,"licenseFileUrl":6,"sourceHtmlUrl":195,"categories":196,"keywords":197,"publishedAt":19},"01M7E1GQ92CWFMGH53KSSE4WER","https:\u002F\u002Fgithub.com\u002Flinyouxue\u002Fskill-repair-benchmark\u002Fblob\u002Fmain\u002Fjpg-ocr-stat\u002Fresult\u002Fskilltta-claude4.7\u002Fgroup_03\u002Frun_02\u002Finputs\u002Fskills\u002Fimage-ocr\u002FSKILL.md",[],[]]