Grasp AI Logo
Back to Blog
Cloud Strategy

SQL to Insights in Minutes: How Copilot in Data Factory Changes the ETL Game

Leon Godwin
10 March 2026

The Challenge

Every organisation has the same data problem. Raw SQL sits in databases, data lakes, and warehouses. Getting it from "stored" to "useful" means building ETL pipelines — mapping transformations, writing SQL joins, configuring data flows. It's skilled work that takes hours, sometimes days.

And there's never enough data engineers to go around.

The business team knows what insights they need. They can describe the transformation in plain English: "Take last quarter's sales data, join it with the customer segmentation table, filter for enterprise accounts, and give me a summary by region." But translating that into a working pipeline has always been the bottleneck.

Microsoft Fabric's Data Factory workload now has Copilot built in. And it's changing who can build data pipelines and how fast they can do it.

What's Changed

Copilot in Data Factory works with Dataflow Gen2 — Microsoft Fabric's data transformation layer. Instead of manually configuring each step of a data flow, you describe what you need in natural language, and Copilot generates the transformation logic.

The Data Exposed episode with Anna Hoffman and Penny from the Data Factory team showed this live. Describe your data source, tell Copilot what transformations you need, and it builds the pipeline. Not a template. Not a suggestion. A working data flow based on your actual data.

Under the hood, Copilot operates as a subject-matter expert for Dataflow Gen2. It understands the data model, the available transformations, and the output format. It can handle complex operations: joins across multiple tables, conditional transformations, type conversions, and aggregations. The kind of work that used to require a data engineer with detailed knowledge of the platform.

This matters for two distinct audiences. Professional data engineers get faster iteration — instead of building flows step by step, they describe the intent and refine the output. Citizen data wranglers — business analysts, operations teams, finance — can now create data transformations without writing SQL at all.

The "minutes not hours" promise isn't marketing. When the alternative is manually configuring each transformation step, having Copilot generate a starting point and then tweaking it shaves significant time off the pipeline development cycle.

Getting Started

Before you start, a few prerequisites. Your Fabric capacity needs to be F2 or P1 or higher — Copilot isn't available on trial SKUs. Your tenant admin must enable the Copilot setting in the Fabric Admin portal. And if your capacity is outside the US or EU, the admin will also need to enable cross-region data processing.

Once enabled, here's the practical workflow:

  1. Open a Dataflow Gen2 in Fabric Data Factory. The Copilot panel appears alongside the flow designer.

  2. Describe your data transformation in natural language. Be specific: "Connect to the Sales database, pull the Orders table, join with Customers on CustomerID, filter where OrderDate is in the last 90 days, and aggregate total revenue by Region."

  3. Review the generated flow. Copilot creates the transformation steps, but you should always validate the output. Check join conditions, filter logic, and aggregation methods. AI-generated pipelines need human review before production use.

  4. Iterate. Ask Copilot to modify specific steps: "Change the date filter to last 180 days" or "Add a column that calculates year-over-year growth." Each refinement updates the flow without starting from scratch.

  5. Deploy and schedule. Once validated, deploy the dataflow and configure your refresh schedule as normal.

For the full walkthrough with live demos, the Data Exposed episode covers real scenarios end to end. Microsoft's official documentation covers configuration requirements and detailed capabilities.

What This Means

This is part of a bigger pattern. Microsoft is embedding Copilot across the entire data platform: Azure SQL Database has Copilot for query writing, Power BI has Copilot for report generation, and now Data Factory has Copilot for pipeline construction. The end-to-end data workflow — from ingestion through transformation to visualisation — is becoming AI-assisted at every step.

For IT leaders evaluating their data platform strategy, this changes the economics. The bottleneck isn't capacity or compute anymore. It's the human expertise required to build and maintain pipelines. Copilot doesn't replace data engineers, but it extends their reach and opens the door for non-specialists to create data transformations that previously required deep platform knowledge.

The organisations that move fastest will be the ones that treat Copilot as a productivity multiplier for their existing teams, not a replacement for data engineering skills.

Leon Godwin, Principal Cloud Evangelist at Cloud Direct