November 18, 2023 6 min read

Microsoft Fabric is GA: What We Learned from Six Months of Preview

Microsoft Fabric Azure Ignite Data Engineering Analytics

Microsoft Ignite 2023 brought the announcement we’ve been waiting for: Microsoft Fabric is now generally available. After more than two years in development and six months in public preview, Fabric is ready for production workloads. Here’s what I’ve learned and what GA means for data teams.

The Numbers Speak

During preview:

25,000 organizations adopted Fabric
67% of Fortune 500 companies started using it
84% of organizations use 3+ workloads (not just Power BI)

These numbers suggest Fabric isn’t just hype - organizations are finding real value in the integrated platform.

What’s GA vs. Still Preview

Generally Available:

OneLake and Lakehouse
Data Factory (Pipelines and Dataflows Gen2)
Data Warehouse
Power BI in Fabric
Data Engineering (Spark notebooks and jobs)
Basic Real-Time Analytics

Still in Preview:

Copilot for Fabric (AI assistance)
Data Activator (event-driven automation)
Some Real-Time Analytics features
Mirroring (replication from external databases)

For production workloads, stick to the GA features. Preview features are improving rapidly but may change.

Pricing Clarity

Microsoft announced reservation pricing that can save up to 40.5% over pay-as-you-go:

Capacity	Pay-As-You-Go	1-Year Reserved
F2	~$262/month	~$175/month
F4	~$525/month	~$350/month
F64	~$8,400/month	~$5,600/month

Key pricing insights:

One meter for all workloads: Spark, SQL, Dataflows, Power BI - all draw from the same capacity
Burst capability: Exceed your capacity temporarily (with throttling)
Pause/resume: Stop billing when not in use (for dev/test environments)
OneLake storage: Separate charge, ~$0.023/GB/month

Copilot in Fabric Preview

The most exciting preview announcement is Copilot integration. AI assistance is coming to:

Power BI:

User: Create a report showing sales trends by region with year-over-year comparison

Copilot: [Generates a multi-page report with appropriate visualizations,
         filters, and DAX measures for YoY calculations]

Data Engineering:

# In a notebook, ask Copilot:
# "Transform this sales data: remove nulls, standardize dates, add fiscal year column"

# Copilot generates:
df = df.dropna(subset=['amount', 'customer_id'])
df = df.withColumn('sale_date', to_date(col('sale_date'), 'yyyy-MM-dd'))
df = df.withColumn('fiscal_year',
    when(month(col('sale_date')) >= 7, year(col('sale_date')) + 1)
    .otherwise(year(col('sale_date'))))

Data Factory: Natural language to pipeline creation - describe what you want to move and transform, Copilot generates the pipeline.

This is still preview and requires appropriate licensing, but the potential is significant.

Lessons from Six Months of Preview

1. OneLake Shortcuts are Powerful

We migrated a client from Azure Data Lake + Synapse to Fabric without moving data:

# Create shortcut to existing ADLS data
# No data movement, immediate access in Fabric

# In Fabric portal:
# Lakehouse > Get data > New shortcut > Azure Data Lake Storage Gen2
# Point to existing containers

# Immediately available for:
# - Spark processing
# - SQL queries via SQL endpoint
# - Power BI reports

2. Start with Lakehouse, Not Warehouse

The Lakehouse is more flexible:

Write with Spark or Dataflows
Query with SQL via auto-generated endpoint
Direct Power BI connectivity
Full Delta Lake capabilities

Reserve the Warehouse for scenarios requiring:

Traditional T-SQL workloads
Specific SQL Server compatibility
Teams that only know SQL

3. Capacity Planning Requires Monitoring

Fabric’s single capacity model means workloads compete. We’ve seen:

Heavy Spark jobs throttling Power BI queries
Large dataflow refreshes impacting report performance
Need for workload management

# Monitor capacity usage via Fabric Admin REST APIs
import requests
from azure.identity import DefaultAzureCredential

credential = DefaultAzureCredential()
token = credential.get_token("https://api.fabric.microsoft.com/.default")

headers = {"Authorization": f"Bearer {token.token}"}

# Get capacity metrics via Admin API
capacity_id = "your-capacity-id"
url = f"https://api.fabric.microsoft.com/v1/admin/capacities/{capacity_id}"

response = requests.get(url, headers=headers)
capacity_info = response.json()

# For detailed metrics, use the Fabric Capacity Metrics app
# or query the metrics workspace
print(f"Capacity state: {capacity_info.get('state')}")

# Alert if capacity is throttling
if capacity_info.get("state") == "Throttled":
    print("WARNING: Fabric capacity is throttling - consider scaling up")

4. Governance is Built-in but Requires Setup

OneLake provides unified governance, but you need to configure:

Workspace roles and permissions
Sensitivity labels
Endorsement for trusted content
Lineage tracking

Don’t assume default settings are sufficient for enterprise compliance.

Migration Patterns

From Power BI Premium:

Easiest migration - enable Fabric on existing capacity
Lakehouse can supplement existing datasets
Gradual adoption of additional workloads

From Azure Synapse Analytics:

OneLake shortcuts to existing ADLS data
Migrate pipelines to Fabric Data Factory
Consider hybrid: Synapse Serverless + Fabric Lakehouse

From Databricks:

Unity Catalog data accessible via shortcuts
Consider keeping Databricks for ML workloads
Use Fabric for Power BI and downstream analytics

What I’m Excited About

Real-Time Intelligence: The combination of KQL databases, Real-Time hub, and Data Activator creates an end-to-end streaming platform. For IoT and operational analytics, this is compelling.

Copilot Potential: AI assistance across the entire data lifecycle - from ingestion to visualization - could dramatically accelerate development.

Simplified Operations: One capacity, one admin experience, one governance model. For organizations drowning in Azure service sprawl, Fabric offers consolidation.

What I’m Watching

Performance at Scale: GA doesn’t mean battle-tested. Large workloads will reveal limitations.

Copilot Accuracy: AI-generated code and reports need careful validation. Hallucinations in data pipelines have real consequences.

Multi-Cloud Story: Fabric is deeply Azure-native. Organizations with multi-cloud strategies need hybrid approaches.

Getting Started with GA

Assess Current State: Inventory your existing Azure data services
Identify Quick Wins: Power BI Premium workloads are easiest to migrate
Pilot with Lakehouse: Start a new project in Fabric Lakehouse
Monitor Capacity: Understand your consumption patterns before committing
Train Teams: The unified experience requires updated skills

Conclusion

Microsoft Fabric GA is a milestone. It’s not the end of the journey - many features are still preview, and real-world production experience will reveal areas for improvement. But for organizations committed to the Microsoft data platform, Fabric represents the future.

The question is no longer whether to adopt Fabric, but when and how.