microsoft-fabric-pipeline-operations
Create, update, run, schedule, and monitor Microsoft Fabric Data Factory pipelines through the Fabric REST API. Use for Fabric pipeline items (DataPipeline) and Copy Jobs, pipeline definition JSON payloads, pipeline activities (Copy, Notebook, Lookup, ForEach), triggering a pipeline run via the job scheduler API, monitoring pipeline runs and activity runs, or setting up pipeline schedules in Microsoft Fabric.
- License 许可
- Apache-2.0
- 最近更新
- 2026-10-27
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- microsoft-fabric-pipeline-operations
- description
- Create, update, run, schedule, and monitor Microsoft Fabric Data Factory pipelines through the Fabric REST API. Use for Fabric pipeline items (DataPipeline) and Copy Jobs, pipeline definition JSON payloads, pipeline activities (Copy, Notebook, Lookup, ForEach), triggering a pipeline run via the job scheduler API, monitoring pipeline runs and activity runs, or setting up pipeline schedules in Microsoft Fabric.
03 SKILL.md 正文
Microsoft Fabric Pipeline Operations
Outcome
Manage Fabric Data Factory pipelines as code: create or update pipeline item definitions via REST, trigger runs, monitor them to a terminal state, and attach schedules — with approval for every mutation.
Setup
- Follow the
microsoft-fabric-api-operationsskill first: Entra ID identity, token scopehttps://api.fabric.microsoft.com/.default, base URLhttps://api.fabric.microsoft.com/v1, LRO polling, and 429 handling all apply. Never place connection credentials or tokens in pipeline JSON or artifacts — Fabric pipelines reference connection objects and workspace identities, not inline secrets. - Verify the workspace and capacity:
GET /v1/workspacesand confirm the target workspace ID. Pipeline creation requires a workspace on a Fabric capacity and Member or Admin workspace role for writes. - Check item-type support for your identity: the DataPipeline item reference pages state service-principal support; verify before automating with one.
Pipeline definitions as items
- A pipeline is an item of
type: "DataPipeline". Create it:POST /v1/workspaces/{workspaceId}/itemswith{displayName, type: "DataPipeline"}, or with adefinitionpayload for a fully defined pipeline.POSTmay return 202 — poll theLocationheader untilSucceeded. - The definition is a set of base64-encoded parts under
definition.parts[]:pipeline-content.json(the pipeline:properties.activities[]withdependsOn,parameters,variables) plus the.platformmetadata part. Fetch the current definition withPOST /v1/workspaces/{wsId}/items/{itemId}/getDefinition; replace it withPOST .../updateDefinition. AlwaysgetDefinitionbeforeupdateDefinitionso the change is a reviewable diff and the prior definition is the rollback. - Activities:
Copy(data movement with source/sink, translator, staging),Lookup,SetVariable,ForEach,IfCondition,Notebook, plusExecutePipeline. Chained execution usesdependsOn: [{activity, dependencyConditions: ["Succeeded"]}]. - Copy Job: a separate item type (
CopyJob) for managed copy-with-incremental behavior; create it the same way withtype: "CopyJob". Prefer it over hand-built Copy activities when the requirement is simple ongoing data movement.
Run, monitor, schedule
- Trigger on demand:
POST /v1/workspaces/{wsId}/items/{itemId}/jobs/instances?jobType=Pipelinereturns 202 with aLocationheader pointing at the job instance. - Monitor:
GET <Location>(orGET /v1/workspaces/{wsId}/items/{itemId}/jobs/instances/{jobInstanceId}) untilstatusisCompleted,Failed, orCancelled. List recent runs withGET .../jobs/instances. For activity-level detail, use the pipeline run monitoring guidance in the Data Factory docs. - Schedule: create a schedule on the item via the job scheduler schedules API (
POST /v1/workspaces/{wsId}/items/{itemId}/jobs/Pipeline/schedules) withenabled,configuration(cron-type withstartDateTime,endDateTime,localTimeZoneId,times), andowner. New schedules start disabled until approved. - Cancel:
POST .../jobs/instances/{id}/cancel— mutating, requires approval and a reason.
Permissions and safety
- Reads (
getDefinition, item/run lists) are unrestricted. Creates, definition updates, runs, schedule changes, and deletes require a written plan (workspace ID, item ID, definition diff or run parameters) and explicit user approval. Deleting a pipeline item requires confirming no schedule or downstream dependency uses it, and a saved definition for rollback. - Workflow: inspect (get item + definition) → plan (definition diff) → execute (update, poll LRO) → verify (re-read definition; trigger a run and watch it reach terminal state) → rollback (re-
updateDefinitionfrom the saved definition).
Troubleshooting
updateDefinitionfails validation:pipeline-content.jsonmust be valid pipeline JSON and base64-encoded per part; check partpathvalues exactly match (pipeline-content.json,.platform).- Run stuck or never starts: you ignored the 202
Location— poll the job instance; also check capacity throttling (429 withRetry-After). - Copy activity fails on connection: the pipeline references a Fabric connection ID that does not exist in this workspace or the caller lacks access; list connections and rebind before rerunning.
- Run shows
Failedwith no detail at the job level: fetch activity-run details per the monitoring doc; job-instance status alone hides the failing activity. - Schedule fires at the wrong time:
localTimeZoneIdmust be a valid Windows/IANA zone; verify with a get-schedule call after creation.
References
- Fabric pipelines via REST (Data Factory): https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-rest-api
- Create Data Pipeline: https://learn.microsoft.com/en-us/rest/api/fabric/datapipeline/items/create-data-pipeline
- Update pipeline definition: https://learn.microsoft.com/en-us/rest/api/fabric/datapipeline/items/update-data-pipeline-definition
- Data pipeline item reference: https://learn.microsoft.com/en-us/rest/api/fabric/datapipeline/items
- Run on-demand item job: https://learn.microsoft.com/en-us/rest/api/fabric/core/job-scheduler/run-on-demand-item-job
- Monitor pipeline runs: https://learn.microsoft.com/en-us/fabric/data-factory/monitor-pipeline-runs
- Item definition format: https://learn.microsoft.com/en-us/rest/api/fabric/articles/item-management/definitions/item-definition-overview
Shared resources
Load only if the task needs them; the skill is self-contained without them.
../../templates/pipeline-design-specification.md../../templates/incident-runbook.md../../references/reliability-and-slo-guide.md
Completion check
getDefinition matches the approved pipeline definition, the triggered run reached Completed (or a diagnosed terminal state) with activity-level evidence, any schedule exists with the approved times and timezone, and no credentials appear in the pipeline JSON or any artifact.
04 同仓库更多 Skills
twenty-year-land-plan
Apply the Bren discipline of evaluating every land and development decision on a twenty-year horizon, with a comprehensive plan for how the land, the community, and the surrounding public infrastructure evolve together. Deals optimized for a five-year exit are structurally different from deals that compound across generations; insist on knowing which kind you are making. Use when evaluating land, entitlements, master-planned development, or phasing decisions, and when separating five-year exit deals from generational holds.
thematic-gut-check
Apply the Gray discipline of identifying the structural theme behind every real estate deal, testing where that theme sits in its capital-flow lifecycle, and insisting that deals without a named theme — or deals late in a crowded theme — clear a higher bar. A mediocre deal in a strong theme usually beats a clever deal in a weak one. Use when naming the structural thesis behind a deal, judging whether that theme is early or crowded, and setting the bar the deal must clear.
supply-pipeline-gut-check
Apply the Zell discipline of grounding every real estate thesis in the local supply pipeline. Demand counts over adjectives — units under construction, units permitted, units proposed — and convert them into a forward absorption schedule. A deal that ignores the pipeline is a bet on a market you have not actually studied. Use when validating a market thesis, forecasting absorption and rent growth, or counting units under construction, permitted, and proposed.
ruin-scenario-mapping
Apply the Taleb discipline of identifying, for every real estate deal, the specific chain of events that produces ruin — permanent, unrecoverable capital loss — and verifying that no single such chain has non-trivial probability. Deals with any plausible ruin path must be restructured or abandoned. Use when sizing leverage, testing refinancing, covenant, and liquidity risk, or checking whether any single failure chain can permanently wipe out the position.
revenue-cost-architecture
Design a venture's revenue streams, pricing logic, cost structure, and unit economics so the business model has a coherent theory of value capture, not just value creation. Use after the value proposition and customer segment are set and before financial projections, when choosing a pricing model, calculating contribution margin or CAC payback, or checking whether the unit economics hold at scale.
replacement-cost-stress-test
Apply the Buffett discipline of pricing every real estate asset against the cost to rebuild it from scratch. The ratio of purchase price to replacement cost sets a ceiling on long-run downside: below replacement cost, new supply cannot profitably undercut you; at or above it, you are buying a liability disguised as an asset. Use when setting a purchase price, judging downside protection, or testing how exposed an asset is to newly built competing supply.