21 min read
Fabric Data Agent Accuracy Is a Distribution, Not a Number
I asked a Fabric data agent 24 questions in three wordings, ten times each, on real AEMO data. It scored 82%, but only 39% of wordings were always right.
Microsoft Fabric
301 articles
6 min read
The Small Files Problem: How Tables End Up With Two Million Files
Small files come from simple arithmetic, and every reader pays a cost per file. Where they come from, how to measure them, and how to compact them safely.
9 min read
Fabric's Data Engineering Agent: Set Guardrails Before Autonomy
Fabric's preview data engineering agent runs with the launching engineer's permissions. What to delegate first, which settings to pick, and what stays human.
7 min read
BigQuery Mirroring Is GA: Zero-ETL Multicloud Has a Bill Too
Fabric mirroring for BigQuery is GA, but it moves cost to Google: CDC compute, egress, keyless reseeds and schema ownership decide when to mirror or copy.
9 min read
Fabric IQ Goes GA in Copilot: Your Semantic Model Is Now an Agent API
Fabric IQ is GA in Copilot Chat, Cowork and MCP. Treat each semantic model as an agent endpoint: RLS, OLS, metadata contracts and default-on access.
7 min read
Partition Pruning: The Fastest Data Is the Data You Never Read
A query for one day should not read three years. How partition pruning works, the five ways it silently breaks, and how to pick partition columns.
7 min read
Reading a Spark Query Plan: Ten Minutes That Explain a Two-Hour Job
The physical plan shows what Spark will actually do: shuffles, join strategy, pruning, Python UDFs. How to read one, and which plans should worry you.
8 min read
Workspace Identity for Eventstream Sources: Retire the Shared Access Key
Eventstream can now read Azure Event Hubs with workspace identity. How to move off SAS keys, the RBAC each side needs, and when a service principal still wins.
7 min read
Data Skew: Why 199 Tasks Finish in Seconds and One Takes an Hour
One hot key can make one task do a third of a job, and more nodes will not help. How to spot skew in the Spark UI, find the keys, and fix it properly.
8 min read
GPU-Accelerated Fabric Warehouse: How to Read the 7x Claim
Fabric Data Warehouse GPU query acceleration is in preview. Why the 7x benchmark won't predict your bill, and how to run a fair A/B test on your own queries.
8 min read
Fabric Data Agents Move to the Responses API: What Consumers Must Change
Fabric data agents now use GPT-5.1 and a Responses API client. What notebook, Foundry and app callers must change, and why you need a full eval re-run.
7 min read
PBIR Is Now the Default: What Changes for Report Source Control
Developer mode and PBIP are GA and PBIR becomes the default report format. Why that makes report review realistic, what breaks, and how to stage the move.
8 min read
Fabric Runtime 2.0 Is GA but Not Default: An Upgrade Plan
Fabric Runtime 2.0 is GA and set to become the default in late September. A plan to inventory, test libraries and Delta features, and pin runtimes first.
8 min read
Converting Direct Lake Tables to Import: Designing Hybrid Models
Power BI now converts single Direct Lake tables to Import in web modelling, making storage mode a per-table choice with real refresh and governance costs.
9 min read
Event-Driven Refresh for Materialized Lake Views: Schedules Still Win
Event-triggered refresh for Fabric materialized lake views trades predictable capacity and clean ordering for freshness. Where events fit and where they don't.
11 min read
Fabric Item CI/CD Is Mostly Solved. Your Tenant Isn't
Git integration and fabric-cicd ship Fabric items well. Workspaces, capacity, domains, roles and tenant settings still need their own source of truth.
9 min read
Auditing a Fabric Tenant on a Schedule with the Scanner and Admin APIs
Use a read-only service principal, the scanner APIs and the Fabric admin APIs to detect governance drift in a Fabric tenant, without handing it write access.
12 min read
Whose Identity Reaches the Target? OneLake Shortcut Security
How identity flows through OneLake shortcuts: passthrough vs delegated, SQL endpoint access modes, the Direct Lake over SQL exception, and lost policy.
8 min read
Rerun From Failed Activity in Fabric: Design for the Second Run
What a Fabric pipeline rerun actually runs again, why utcNow() and run IDs break recovery, and how to make rerun from failed activity safe.
8 min read
Findable by Design: Workspace Choices That Shape the OneLake Catalog
How workspace splits, item names, descriptions, endorsement and domains decide what people find in the OneLake catalog, plus a findability test you can script.
9 min read
Materialized Lake Views as a Silver Flow: What to Hand Over
Now that Fabric materialized lake views are GA: which raw-to-silver steps I'd move onto them, how to keep rejected rows, and where notebooks still win.
7 min read
Write-Audit-Publish in Fabric: Stop Bad Data Before It Ships
Why many data quality root-cause fixes come down to where the check runs, and how to build a write-audit-publish boundary on a Fabric lakehouse.
8 min read
Real-Time Dashboards in Git: Reviewable Changes and a Way Back
Put Fabric Real-Time Dashboards under Git so tile and query changes are reviewed, validated before Update from Git, and reverted cleanly when they break.
9 min read
Publish Only Verified Loads with Fabric Warehouse Snapshots
A write-audit-publish pattern for Fabric Data Warehouse: load, run SQL checks, then roll a warehouse snapshot forward so analysts only see verified data.
9 min read
Fabric Pipeline Deadlines: Alert on a Late Run Before Anyone Asks
How to catch a slipping Fabric pipeline before its deadline: alert on missing and late runs, not just failures, using job events, KQL and the jobs API.
8 min read
Sensitivity Labels in OneLake: Decide Item Boundaries Before Scale
Fabric sensitivity labels apply per item, not per table, so item boundaries, defaults, inheritance and protection policies need designing before OneLake grows.
8 min read
Changing a Shared Fabric Table Without Breaking Its Notebooks
How to version a Fabric lakehouse table contract once several notebooks depend on it: change classes, a writer guard, pinned readers and safe renames.
9 min read
Blameless Reviews for Data Incidents: Ask Which Control Failed
How to run blameless reviews for data quality incidents: record who detected it, which control was missing, and turn each review into a tracked change.
9 min read
Pricing a Threshold: Tying Fabric Activator Rules to Business Impact
Set Fabric Activator and KQL alert thresholds from the cost of a miss versus the cost of acting, then tier actions by the value at risk, not the raw reading.
8 min read
Fabric Warehouse Tables or Lakehouse Shortcuts: Where Each Table Lives
A placement rule for Fabric: when data belongs in a Warehouse table you own, and when reading a lakehouse shortcut through the SQL analytics endpoint is enough.
8 min read
Readable Fabric Pipelines: Names, Boundaries and Config Off the Canvas
How to keep Fabric pipelines readable as they grow: naming before the first Git commit, child pipelines with clear contracts, and variable libraries for config.
8 min read
Feeding AI Teams Through OneLake Shortcuts Without Over-Sharing
A passthrough OneLake shortcut adds speed but no boundary: how to give data science teams curated Fabric features without opening the whole lakehouse.
8 min read
Ingestion vs Curation in Fabric: Where I Draw the Workspace Line
Why I split Fabric ingestion and curation at the landing table, when that means separate workspaces joined by shortcuts, and when one lakehouse is enough.
7 min read
Failure Budgets for Data Products: SLOs for Fresh, Correct Tables
Measure data product reliability in good hours, catch bad data at the publish boundary, and let a failure budget decide when pipeline changes stop.
9 min read
Store Changes, Not Readings: Shaping Noisy Streams in an Eventhouse
Use update policies, materialized views and a KQL state-change function in a Fabric Eventhouse to strip duplicates, chatter and flapping before anyone queries.
8 min read
When Two Fabric Reports Disagree: A Metric Drift Runbook
A triage runbook for disputed numbers in Fabric: check definition, timing, filters and logic in order, using time travel, snapshots and Semantic Link.
8 min read
Fabric Pipeline Latency: Find Where the Minutes Actually Go
How to split a slow Fabric pipeline run into start delay, queueing, Spark start-up, copy transfer and idle gaps, with KQL and the activity runs API.
8 min read
OneLake Shortcuts Are Dependencies: Inventory Them Before Scale
Treat every OneLake shortcut as a cross-team dependency: placement, identity, write paths, environment variables and a REST inventory of what points where.
8 min read
When a Slow Fabric Lakehouse Is Really a Raw Zone Problem
Why slow Fabric silver loads often trace back to the raw zone: landing layout, header checks, small files and bronze table maintenance, with PySpark examples.
7 min read
Pruning Data Quality Checks: Keep, Demote, or Delete
How to cut an inherited data quality rule set down to the checks that change decisions, using failure history, ownership and an explicit retirement path.
7 min read
Noise or a Real Change? Three KQL Checks Before Anyone Gets Paged
Use KQL baselines, week-over-week comparison and diffpatterns in an Eventhouse to tell a meaningful operational change from ordinary noise, and to explain it.
8 min read
One Metric, One Home: Ownership Rules That Stop Drift in Fabric
How to stop metric drift in Fabric by giving every metric one home layer and one named owner, then enforcing both rules with schemas, grants and versions.
8 min read
Fabric Pipeline Failure Paths: Catch the Error, Then Fail the Run
How dependency conditions decide a Fabric pipeline's final status, why try-catch hides failures, and how to log, alert and still fail the run honestly.
8 min read
Keeping OneLake Clean: Design the Workspace Lifecycle Before Scale
How to stop Fabric workspace and item sprawl before it starts: lifecycle states, domains, tags, endorsement, retention settings and a weekly hygiene audit.
8 min read
Break Your Silver Layer on Purpose: Bad-Input Tests in Fabric
Why I keep a library of deliberately broken raw files for every Fabric bronze-to-silver transform, and how to run them as plain PySpark tests before release.
8 min read
Two Tickets per Data Incident: Contain First, Fix the Cause Second
Why every data quality incident should produce two pieces of work, a containment action and a root-cause fix, and how to run both on a Fabric lakehouse.
8 min read
Alerts People Don't Mute: Designing Activator Rules in Fabric
How to design Fabric Activator and KQL queryset alerts that fire on transitions, carry an owner and a next step, and stop teams muting the channel.
8 min read
SQL in Fabric: Give Analysts a View Layer, Not Your Tables
How I design an analyst-facing schema in Fabric Data Warehouse: views as the contract, permissions scoped to it, and SQL checks that catch breaks first.
8 min read
Fabric Data Movement: Design for the Half-Finished Copy
How to design Fabric Copy jobs and copy activities so a failed, partial or repeated copy never leaves duplicates, gaps or a watermark that moved too far.
9 min read
Fast Access Without Eroding OneLake Boundaries in Fabric
Why adding people as Fabric workspace Contributors is the access shortcut that erodes OneLake boundaries, and a faster pattern that keeps them intact.
9 min read
Who Gets Told? Owners, Error Codes and Runbooks in Fabric
A table contract is half the job: assigning owners per layer, collapsing load outcomes to three, and mapping Fabric pipeline failures to runbooks.
7 min read
Five Data Quality Signals for the Data-to-AI Handoff
Why I'd track five quality signals per feature table at the data engineering to AI handoff, and how to compute them in a Fabric notebook without a new tool.
8 min read
Why Real-Time Dashboards Rot After Week One, and How to Stop It
Rules for Fabric Real-Time Dashboards that stay trusted: tiles with owners, visible data freshness, refresh costs you can afford, and thresholds in tables.
7 min read
Fabric Warehouse: Decide Fact Table Grain Before You Tune Anything
Why fact table grain, not clustering or statistics, sets the performance ceiling in Fabric Data Warehouse, and how to declare and test it in T-SQL.
9 min read
Fabric Daily Loads: Gate on Data Readiness, Not the Clock
How to remove brittle dependencies from daily Fabric loads with one orchestrating pipeline, readiness checks, idempotent writes and deliberate retries.
8 min read
OneLake Table Contracts: Owners, Schemas and Failure Modes Up Front
Why OneLake governance has to be designed before scale: table contracts, schema-enabled lakehouses, explicit owners and deliberate failure modes in Fabric.
9 min read
Table Contracts in a Fabric Lakehouse: Where Raw Becomes a Product
Four decisions for a messy Fabric raw zone: contracts at silver, Delta constraints, quarantined rows with failed batches, and lakehouse schemas.
8 min read
Equipment Monitoring on Fabric Real-Time Intelligence, End to End
One sensor-monitoring design on Fabric Real-Time Intelligence: where to filter, enrich, aggregate and alert, and what Eventhouse retention costs you.
8 min read
KQL for SQL People: The Query Language Azure Already Runs On
Why KQL is a baseline Azure skill, how its pipe model differs from SQL, and the operators, join defaults and has-vs-contains traps that matter most.
7 min read
Delta Lake in Fabric: Not Every Engine Reads Every Table
Every Fabric engine speaks Delta Lake, but not the same features. How to use the interoperability matrix so Spark tables still work in SQL and Power BI.
9 min read
Leaving ADF and Databricks for Fabric: What Held Up
What consolidating Azure Data Factory, Databricks, Azure SQL and Power BI onto Microsoft Fabric delivered in production, and where the seams still show.
8 min read
Running Data Pipelines in Production: Reruns, Late Data and Drift
Eight years of data pipelines taught me to plan for reruns, late data and schema drift. The habits that matter, with Delta MERGE and Fabric examples.
9 min read
Synapse to Fabric Migration: 5 Things I Wish I'd Known
Our Synapse to Fabric migration took twice as long as planned. Five lessons on scope, notebook refactoring, Direct Lake, capacity monitoring and training.
8 min read
From F64 to F32: Three Fabric Sizing Mistakes and the Fixes
Why we moved Microsoft Fabric from F64 to F32: the three sizing mistakes, the monitoring and alerts we added, and when going smaller is the wrong call.
9 min read
Databricks or Fabric in 2026: Choosing Your Primary Platform
A practical decision guide for picking Azure Databricks or Microsoft Fabric as your primary data platform in 2026, and when running both is worth the cost.
9 min read
Microsoft Fabric Reality Check: What Holds Up and What Hurts
An honest look at running Microsoft Fabric on an F64: what OneLake and Direct Lake deliver, how throttling really works, and what the bill looks like.
2 min read
November 2025 Recap: Ignite Announcements and AI Progress
Azure AI Foundry unified the AI development experience, combining model catalog, deployment tools, and prompt management into a single platform. This…
1 min read
Data Governance in Microsoft Fabric: Purview Integration and Compliance
Organizations struggle with data sprawl, unclear ownership, inconsistent definitions, and regulatory requirements. A unified governance approach addresses…
1 min read
Building Data Lakehouses: Medallion Architecture Best Practices
Each layer serves a specific purpose in the data refinement pipeline. Bronze captures raw data, silver cleans and conforms it, and gold delivers…
1 min read
Ignite 2025 Day 2: Microsoft Fabric Innovations and Real-Time Intelligence
These Fabric updates demonstrate Microsoft's commitment to building a truly unified data platform that handles real-time and batch workloads while…
1 min read
Real-Time Data Streaming with Microsoft Fabric Eventstreams
Batch processing introduces latency between data generation and insight availability. For fraud detection, operational monitoring, and customer engagement…
1 min read
Implementing Feature Stores with Microsoft Fabric OneLake
Without a feature store, teams often duplicate feature engineering work across projects. Features computed for training may differ from inference, causing…
1 min read
Delta Lake Deep Dive: Time Travel and Schema Evolution in Fabric
Every write operation to a Delta table creates a new version. This transaction log enables point-in-time queries and rollback capabilities essential for…
1 min read
Implementing Data Quality Checks in Microsoft Fabric Lakehouses
Poor data quality leads to incorrect business decisions, failed ML models, and eroded trust in analytics platforms. A proactive approach to data validation…
1 min read
Implementing Retrieval-Augmented Generation with Microsoft Fabric
Use Fabric notebooks to create interactive RAG applications that combine data exploration with AI-powered question answering over enterprise datasets.
1 min read
Microsoft Fabric Capacity Planning: Right-Sizing Your Data Platform
For development and test environments, implement auto-pause policies to reduce costs during inactive periods. Monitor utilization patterns and adjust…
1 min read
Microsoft Fabric Notebooks: Collaborative Data Science at Scale
Fabric notebooks provide a seamless path from exploration to production, with built-in governance, collaboration, and scalable compute.
1 min read
Microsoft Fabric Data Activator: Event-Driven Data Alerts
Data Activator monitors data streams and triggers actions based on conditions you define. It connects to Power BI reports, Eventstreams, and Fabric data…
1 min read
Microsoft Fabric Lakehouse: Medallion Architecture Implementation
Bronze holds raw ingested data. Silver contains cleansed and conformed data. Gold presents business-level aggregates ready for consumption.
1 min read
Microsoft Fabric Real-Time Analytics: Streaming Data Pipelines
Eventhouse is the storage layer optimized for time-series and streaming data. It automatically indexes data for lightning-fast queries.
1 min read
AI-Powered Data Exploration with Microsoft Fabric Copilot
Ask questions about your data in natural language and get executable code. Generate complex SQL from natural language descriptions.
1 min read
Building Data Pipelines with Microsoft Fabric Data Factory
Data Factory pipelines use a visual designer with activities that can be chained together.
1 min read
Medallion Architecture in Microsoft Fabric: Bronze, Silver, Gold Explained
Bronze tables store data exactly as received from source systems, preserving the original format for replayability and debugging.
1 min read
Data Quality for AI: Ensuring Clean Data for ML and LLM Applications
Quality encompasses completeness, accuracy, consistency, timeliness, and validity. Each dimension requires specific checks.
1 min read
Real-Time Data Processing with Microsoft Fabric Eventstreams
Eventstreams support multiple source types including Azure Event Hubs, Azure IoT Hub, and custom applications. The visual designer makes it easy to…
1 min read
Apache Iceberg on Microsoft Fabric: Open Table Format Integration
Iceberg provides ACID transactions, schema evolution, time travel, and partition evolution for data lakes. Its format-agnostic design works with Spark…
1 min read
Microsoft Ignite 2025 Preview: AI and Data Platform Expectations
Ignite remains the premier event for understanding Microsoft's enterprise AI direction. Planning your learning path around expected announcements helps…
1 min read
Delta Lake Optimization: VACUUM, OPTIMIZE, and Z-ORDER Explained
VACUUM removes data files no longer referenced by the Delta log. Without regular vacuuming, your storage costs grow unbounded as old file versions accumulate.
1 min read
Microsoft Fabric Lakehouse: Unified Analytics for the Modern Data Stack
A Lakehouse stores data in open formats like Delta Lake while providing SQL query capabilities, ACID transactions, and schema enforcement. In Microsoft…
1 min read
Power BI Copilot: Natural Language Analytics for Business Users
Use descriptive table and column names that match business terminology. Add measure descriptions explaining what each calculation represents. Create…
1 min read
Data Lakehouse Architecture with Microsoft Fabric: Medallion Pattern Implementation
Use Delta Lake constraints and expectations to ensure data quality at each layer. Fabric's Data Quality monitoring provides visibility into quality metrics…
1 min read
Fabric Data Pipelines: Orchestrating ML Feature Engineering at Scale
For real-time inference, query the feature store directly from your model serving endpoint. Fabric's query optimization ensures millisecond-level feature…
1 min read
Microsoft Fabric Real-Time Intelligence: Streaming Analytics at Scale
Microsoft Fabric's Real-Time Intelligence workload has matured significantly since GA. Today I'm exploring how to build streaming analytics pipelines that…
3 min read
Microsoft Fabric Real-Time Intelligence: Patterns for Streaming Analytics
Microsoft Fabric's Real-Time Intelligence workload has matured significantly since GA. We've implemented several production streaming scenarios, and I want…
1 min read
Microsoft Fabric AI Integration: Unified Analytics and Intelligence
Microsoft Fabric integrates AI throughout the analytics lifecycle from ingestion to insights.
2 min read
Microsoft Fabric Copilot Now Available for All Paid SKUs
This works on F2 and above now. For small businesses, this transforms how reports are built.
1 min read
Fabric AI Deep Integration: Advanced Patterns
Fabric + AI creates a powerful platform for intelligent analytics. Leverage Spark's scale with AI's intelligence.
1 min read
Microsoft Build 2025 Preview: What to Expect for Data and AI
Build 2025 promises to be significant for data and AI professionals. Stay tuned for the actual announcements and be ready to experiment with new capabilities.
1 min read
Microsoft Fabric and AI Integration: Complete Guide
Fabric + AI creates a powerful combination for intelligent analytics. Start with simple enrichments and build toward complex AI-powered workflows.
6 min read
Drafting a Star Schema with GPT-4o and Structured Outputs
Use GPT-4o structured outputs to draft a star schema from requirements, check it with plain code, and generate Fabric Warehouse DDL that respects its limits.
7 min read
Natural Language over a Power BI Model: Let the LLM Pick, Not Write
Answer plain-English questions from a Power BI semantic model by having Azure OpenAI choose measures and filters, then run them with semantic link.
7 min read
Rolling Out a Data Mesh on Fabric: A Phased Plan for 2025
A phased plan for rolling out data mesh on Microsoft Fabric and Purview in early 2025: what to build first, what's still preview, and when to stop.
7 min read
Where Streaming Logic Belongs in Fabric: Eventstream, KQL or Spark
A decision guide for placing filters, enrichment, aggregation and alerts across Eventstream, Eventhouse and Spark in Fabric Real-Time Intelligence.
7 min read
Reviewing Copilot Output in Fabric: A Workload-by-Workload Guide
Where Copilot in Microsoft Fabric stands in January 2025, what each workload's Copilot gets wrong, and how to review its output before it ships.
8 min read
Planning Fabric Work for 2025: What's GA, Preview, and Worth Waiting On
A January 2025 status check on Microsoft Fabric after Ignite 2024: which features are GA, which are preview, and how to sequence this year's work.
1 min read
Fabric Adoption Patterns: Lessons from Enterprise Implementations
Success with Fabric comes from treating it as a platform transformation, not just a technology deployment. Invest in people, process, and governance…
1 min read
Microsoft Fabric Year in Review: From GA to Enterprise Standard
1. Start with a focused use case Don't try to migrate everything at once 2. Embrace lakehouse architecture It's the foundation of Fabric 3. Invest in…
3 min read
November 2024 Wrap-Up: A Transformative Month for Enterprise AI
Fabric AI Skills, Copilot for Microsoft 365, and Copilot Studio all contribute to this trend.
1 min read
OneLake AI Workloads: Running AI Directly on Your Data Lake
OneLake AI Workloads bring AI capabilities directly to your data, eliminating data movement and enabling efficient large-scale AI processing.
1 min read
Fabric and Copilot Studio Integration: Building Data-Driven AI Assistants
The Fabric + Copilot Studio integration makes enterprise data conversationally accessible while maintaining security and governance.
2 min read
Microsoft Ignite 2024 Recap: Key Announcements for Data and AI
This unification simplifies the developer experience and provides a clear path from experimentation to production.
1 min read
Streaming Machine Learning: Training and Inference on Real-Time Data
Streaming ML enables intelligent real-time systems that continuously learn and adapt. Start with robust feature engineering and monitoring before enabling…
1 min read
Real-Time AI in Microsoft Fabric: Patterns and Practices
Real-time AI enables immediate, intelligent responses to streaming data. Start with well-defined use cases and gradually increase complexity.
1 min read
Fabric Eventstream Enhancements: Real-Time Data at Scale
Eventstream enhancements make real-time analytics more powerful and accessible. Start with simple streaming scenarios and progressively add complexity.
1 min read
Fabric Analytics Agents: Automated Data Analysis
Analytics agents reduce manual toil while improving data reliability. Start with monitoring agents for quick wins, then expand to exploration and quality…
1 min read
Microsoft Fabric AI Skills: Natural Language Analytics
AI Skills democratize data access while maintaining governance. Start with well-defined domains and expand based on user feedback.
1 min read
Microsoft Fabric at Ignite 2024: What's New for Data Professionals
The Fabric platform continues to mature into a comprehensive, AI-native analytics solution.
1 min read
DP-600 Exam Overview: Implementing Analytics Solutions Using Microsoft Fabric
The DP-600 certification is your gateway to becoming a recognized Microsoft Fabric expert. Good luck with your exam preparation!
1 min read
Microsoft Fabric Certification Prep: Getting Ready for DP-600
The DP-600 certification demonstrates your expertise in Microsoft Fabric. Focus on hands-on practice and understanding the "why" behind each feature, not…
1 min read
Microsoft Fabric Roadmap: What's Coming in 2024-2025
Stay informed about the Fabric roadmap to plan your data platform strategy effectively. The pace of innovation is rapid, so regular review of announcements…
1 min read
Microsoft Fabric Architecture Evolution: Past, Present, and Future
Microsoft Fabric represents the natural evolution toward unified, AI-powered data platforms. Understanding this evolution helps you make informed decisions…
1 min read
Building a Unified Data Platform with Microsoft Fabric
A unified data platform on Fabric eliminates the complexity of managing multiple systems while providing the flexibility to handle any data workload.
1 min read
Unifying Transactional and Analytical Workloads in Fabric
Unifying Transactional and Analytical Workloads in Fabric
1 min read
HTAP Patterns in Microsoft Fabric: Unifying Transactions and Analytics
HTAP in Microsoft Fabric enables unified transactional and analytical workloads without complex ETL pipelines. Design your schemas with both workloads in mind.
1 min read
PostgreSQL in Microsoft Fabric: What to Expect
PostgreSQL in Fabric will bring familiar PostgreSQL capabilities to the unified data platform. Start preparing now by designing schemas that work well with…
1 min read
SQL Database in Microsoft Fabric: Deep Dive
SQL Database in Fabric brings enterprise-grade transactional capabilities to the unified data platform, with seamless integration to analytical workloads…
1 min read
Fabric Databases Preview: Relational Databases in Microsoft Fabric
Fabric Databases bring operational database workloads into the unified Fabric platform, with automatic integration to OneLake.
1 min read
Microsoft Fabric Updates: October 2024 Announcements
Microsoft announced the preview of relational databases directly within Fabric, bringing transactional workloads into the unified platform.
1 min read
Microsoft Fabric Governance: A Complete Guide
Effective Microsoft Fabric governance requires attention to access control, data quality, cost management, and lifecycle management. This month's…
1 min read
Usage Reporting in Microsoft Fabric
1. Collect comprehensively : Gather data from all sources 2. Aggregate appropriately : Different audiences need different views 3. Automate distribution :…
1 min read
Cost Allocation Strategies for Microsoft Fabric
1. Tag everything : Consistent tagging enables accurate allocation 2. Use multiple methods : Combine direct, shared, and overhead allocation 3. Document…
1 min read
Chargeback Models for Microsoft Fabric
1. Start with showback : Build understanding before chargeback 2. Keep it simple : Complex models create disputes 3. Automate collection : Manual processes…
1 min read
FinOps for Microsoft Fabric
1. Start with visibility : You can't optimize what you can't see 2. Allocate costs : Enable accountability through showback/chargeback 3. Automate policies…
1 min read
Cost Management in Microsoft Fabric
1. Monitor regularly : Daily cost tracking enables early intervention 2. Set alerts : Budget alerts prevent surprises 3. Allocate costs : Understand which…
1 min read
Understanding Smoothing and Bursting in Fabric
Smoothing averages capacity consumption over a time window, allowing brief spikes without immediate throttling.
1 min read
Autoscale in Microsoft Fabric
Autoscale in Microsoft Fabric provides the flexibility to handle variable workloads while optimizing costs. Configure base capacity for typical load, allow…
1 min read
Microsoft Fabric Capacity Optimization
1. Monitor utilization patterns : Understand your usage before optimizing 2. Stagger workloads : Avoid concurrent peaks 3. Right size jobs : Use appropriate…
1 min read
Native Execution Engine in Microsoft Fabric
Native execution bypasses traditional JVM-based Spark execution to run queries using native code optimized for modern CPUs.
1 min read
Spark Optimization in Microsoft Fabric
1. Enable AQE : Adaptive Query Execution handles many optimizations automatically 2. Right size partitions : Target 128MB partitions 3. Broadcast small…
1 min read
Query Performance Tuning in Microsoft Fabric
1. Filter early : Push filters as close to data source as possible 2. Use appropriate joins : Broadcast small tables 3. Enable AQE : Adaptive Query…
1 min read
Z-Order vs V-Order: Choosing the Right Data Ordering
1. Always enable V Order : Low overhead, universal benefits 2. Add Z Order for specific needs : Known filter patterns 3. Limit Z Order columns : Maximum 3 4…
1 min read
V-Order: Microsoft Fabric's Performance Optimization
V-Order is a write-time optimization that physically reorders data within Parquet files to maximize compression and query performance, especially for Direct…
1 min read
Optimized Writes in Delta Lake
1. Enable by default : For most analytical workloads 2. Choose appropriate bin size : 128MB general, 256MB for Direct Lake 3. Combine with auto compact :…
1 min read
File Compaction Strategies in Delta Lake
1. Schedule regular compaction : Daily for high volume tables 2. Use partition aware compaction : Avoid rewriting entire large tables 3. Enable auto…
1 min read
Parquet File Optimization for Analytics
1. Target 128 256MB files : Optimal for most query engines 2. Use Snappy or ZSTD : Best compression/speed balance 3. Sort by filter columns : Improves…
1 min read
Delta Lake Everywhere: The Universal Table Format
Delta Lake provides a reliable, performant foundation for modern data platforms. Its ACID transactions, time travel, and schema evolution capabilities make…
1 min read
OneLake Interoperability: Connecting Your Data Ecosystem
1. Use managed identity : Preferred for Azure services 2. Leverage shortcuts : Don't copy data unnecessarily 3. Optimize file sizes : 128MB 1GB for best…
1 min read
Microsoft Fabric and Databricks: Coexistence Strategies
1. Use shared storage : ADLS as the common data layer 2. Standardize on Delta Lake : Compatible format for both 3. Optimize for readers : Both platforms…
1 min read
Cross-Cloud Analytics with Microsoft Fabric
Most enterprises use multiple clouds: 1. Minimize cross cloud joins : Materialize frequently joined data 2. Use shortcuts wisely : Only for data that…
1 min read
Lakehouse Federation in Microsoft Fabric
1. Define clear domains : Each lakehouse should own specific data 2. Minimize cross domain joins : Materialize frequently joined data 3. Implement…
1 min read
Data Virtualization in Microsoft Fabric
Traditional ETL moves data physically: Virtualization queries in place: Shortcuts create virtual pointers to external data: 1. Use for read heavy workloads…
1 min read
Change Data Capture in Microsoft Fabric
CDC captures row level changes (inserts, updates, deletes) from source systems: The simplest option for supported sources: For more control over the CDC…
1 min read
Real-Time Sync Patterns in Microsoft Fabric
For sub second latency: For minute level latency with relational sources: Combine streaming and batch for different data types: Distribute data to multiple…
1 min read
Open Mirroring in Fabric: Build Custom Data Sources
Open Mirroring provides a standardized way to: Write Delta Lake tables directly to a mirrored database location Maintain the mirrored database experience…
1 min read
Snowflake Mirroring to Fabric: Cross-Cloud Data Integration
Mirroring eliminates the need for complex ETL between these systems.
1 min read
Cosmos DB Mirroring to Fabric: Analytical Queries at Scale
Unlike SQL mirroring which uses change tracking, Cosmos DB mirroring leverages the built in analytical store: 1. Go to your Cosmos DB account 2. Navigate to…
1 min read
Azure SQL Mirroring to Fabric: Complete Setup Guide
1. Navigate to your workspace 2. Click New Mirrored Database 3. Select Azure SQL Database 4. Configure connection: Server name Database name Authentication…
1 min read
Fabric Mirroring GA: Real-Time Database Replication
Mirroring creates a continuously synchronized copy of your operational database in OneLake as Delta Lake tables: As of GA: Azure SQL Database : Full support…
1 min read
Microsoft Fabric June 2024: Month in Review
This month we explored Microsoft Fabric comprehensively. Today I'm summarizing key learnings and best practices from my June 2024 deep dive.
1 min read
Private Endpoints Deep Dive for Microsoft Fabric
Private Endpoints enable secure, private connectivity to Microsoft Fabric. Today I'm taking a deep dive into implementing private endpoints.
1 min read
Network Security in Microsoft Fabric
Network security provides defense in depth for your data platform. Today I'm exploring network security options in Microsoft Fabric.
1 min read
Conditional Access Policies for Microsoft Fabric
Conditional Access enables Zero Trust security by controlling access based on conditions. Today I'm exploring how to implement Conditional Access for…
1 min read
Data Loss Prevention in Microsoft Fabric
Data Loss Prevention (DLP) helps prevent accidental data exposure and ensures compliance. Today I'm exploring DLP implementation in Microsoft Fabric.
1 min read
Sensitivity Labels and Data Classification in Microsoft Fabric
Sensitivity labels help classify and protect data based on its sensitivity level. Today I'm exploring how to implement data classification in Microsoft Fabric.
1 min read
Security Fundamentals in Microsoft Fabric
Security is foundational to any data platform. Today I'm exploring the security architecture and best practices in Microsoft Fabric.
1 min read
External Data Access Patterns in Microsoft Fabric
Accessing external data is essential for hybrid and multi-cloud scenarios. Today I'm exploring external data access patterns in Microsoft Fabric.
1 min read
Data Sharing Patterns in Microsoft Fabric
Data sharing enables collaboration while maintaining security. Today I'm exploring data sharing patterns in Microsoft Fabric.
1 min read
Building a Data Marketplace in Microsoft Fabric
A data marketplace enables data discovery, sharing, and consumption across your organization. Today I'm exploring how to build one using Microsoft Fabric.
1 min read
API-First Data: Treating Data as a Service
API-first data design treats data products as services with well-defined interfaces. Today I'm exploring how to build data APIs in Microsoft Fabric.
1 min read
Data Contracts: Agreements Between Data Producers and Consumers
Data contracts formalize the agreement between data producers and consumers. Today I'm exploring how to implement effective data contracts in your data…
1 min read
Building Data Products in Microsoft Fabric
Data products are the deliverables of a data mesh. Today I'm exploring how to build, manage, and consume data products in Microsoft Fabric.
1 min read
Federated Governance in Data Mesh
Federated governance balances domain autonomy with organizational standards. Today I'm exploring how to implement effective federated governance in…
1 min read
Domain-Driven Design for Data
Domain-Driven Design (DDD) principles apply to data architecture just as they do to software. Today I'm exploring how to design data systems around business…
1 min read
Data Mesh with Microsoft Fabric
1. Clear domain boundaries Define ownership clearly 2. Product thinking Treat data as a product 3. Self serve enablement Empower domain teams 4.…
1 min read
Workspace Governance in Microsoft Fabric
Workspaces are the containers for collaboration in Fabric. Effective governance ensures organization, security, and efficiency. Today I'm covering workspace…
1 min read
Tenant Settings in Microsoft Fabric
Tenant settings control what users can do in your Fabric environment. Today I'm covering the essential settings every admin should configure.
1 min read
Capacity Management in Microsoft Fabric
Effective capacity management is crucial for Fabric performance and cost control. Today I'm diving deep into capacity planning and optimization.
1 min read
Fabric Admin Updates: Governance and Management
Fabric administration has matured significantly. Today I'm covering the latest admin capabilities for managing your Fabric environment effectively.
1 min read
Copilot for SQL in Microsoft Fabric Data Warehouse
Copilot for SQL transforms natural language into T-SQL queries. Today I'm exploring how to leverage AI for data warehouse development in Fabric.
1 min read
Copilot for Notebooks: AI-Assisted Data Engineering
Copilot for Notebooks transforms how data engineers write code. Today I'm exploring practical patterns for leveraging AI assistance in Fabric notebooks.
1 min read
Fabric Copilot Updates: AI-Powered Data Analytics
Fabric Copilot has received significant updates. Today I'm exploring how AI assistants are transforming data work in Microsoft Fabric.
1 min read
Automated Actions in Microsoft Fabric
Automated actions are what make Data Activator powerful. Today I'm exploring all the ways you can respond to data conditions automatically.
2 min read
Reflex Triggers: Building Automated Data Workflows
Reflex triggers are the heart of Data Activator. Today I'm diving deep into building sophisticated automated workflows that respond to your data.
1 min read
Data Activator GA: Automated Actions from Data
Data Activator monitors your data and triggers actions when conditions are met: A Reflex is a Data Activator item containing: Objects : Entities to monitor…
1 min read
Real-Time Dashboards in Microsoft Fabric
Real-time dashboards bring your streaming data to life. Today I'm exploring how to build effective real-time visualizations in Fabric.
1 min read
KQL Querysets in Microsoft Fabric
KQL Querysets are the analysis layer of Real-Time Intelligence in Fabric. Today I'm exploring how to write effective KQL for real-time analytics.
1 min read
Eventhouses in Microsoft Fabric: Deep Dive
Eventhouses are the foundation of Real-Time Intelligence in Fabric. Today I'm exploring their architecture and capabilities in depth.
1 min read
Microsoft Fabric June 2024 Updates: Real-Time Intelligence GA
Real-Time Intelligence is Fabric's answer to streaming analytics, combining the power of Azure Data Explorer with the unified Fabric experience.
1 min read
Power BI April 2024 Updates: AI-Powered Analytics
Power BI continues to evolve with AI-driven features that make data analysis more accessible. Here's what's new in April 2024.
1 min read
Automation in Fabric: Scheduling and Orchestration
Automation is key to maintaining reliable data pipelines. This guide covers scheduling, orchestration, and automation patterns in Microsoft Fabric.
1 min read
Fabric SDK: Python Development for Microsoft Fabric
The Microsoft Fabric SDK provides a Pythonic interface for working with Fabric services. This guide covers installation, configuration, and common…
1 min read
Fabric REST APIs: Programmatic Control of Your Analytics Platform
Microsoft Fabric's REST APIs enable automation and integration with your existing tools. This guide covers the key APIs and common use cases.
1 min read
Fabric CI/CD: Building Deployment Pipelines
Implementing CI/CD for Microsoft Fabric ensures reliable, repeatable deployments of analytics artifacts. This guide covers building end-to-end deployment…
1 min read
Fabric Git Integration: Version Control for Analytics
Microsoft Fabric's Git integration brings modern version control practices to analytics. This guide covers setting up and using Git with Fabric workspaces.
1 min read
Automated Report Generation with AI
AI can transform raw data into polished, narrative reports. This guide covers building automated report generation systems that combine data analysis with…
1 min read
Data Analysis with Copilot: AI-Powered Insights
Microsoft Fabric's Copilot transforms data analysis by enabling natural language interactions with your data. This guide shows how to leverage Copilot for…
1 min read
Fabric Copilot Improvements: Smarter AI Assistance
Microsoft Fabric Copilot has received significant upgrades, making AI-assisted data analysis more powerful and contextually aware. Here's what's new and how…
1 min read
Microsoft Fabric April 2024 Updates: What's New
Microsoft Fabric continues to evolve as a unified analytics platform. April 2024 brings significant updates across data engineering, science, and business…
2 min read
Microsoft Fabric Data Engineering: Best Practices After 6 Months in Production
Avoid over-partitioning. If partitions have < 1GB of data, consolidate. Use Data Pipelines, Not Just Notebooks
8 min read
Data Activator Preview Readiness: Limits, Admin Settings and Risks
A readiness checklist for the Data Activator preview in Fabric: tenant switches, throughput and alert limits, failure modes, and when to wait for GA.
9 min read
Designing Reflex Triggers in Data Activator Without Alert Spam
How Data Activator preview triggers really fire: objects, Is vs Becomes conditions, timers, Power Automate custom actions and the throughput limits to plan for.
7 min read
Keeping KQL Fast on Streaming Data: Query Habits That Matter
The KQL habits that keep queries over streaming telemetry fast in Fabric KQL databases and Azure Data Explorer, and the shortcuts that quietly slow them down.
8 min read
Fabric Eventstream Patterns: Deciding Where Stream Logic Lives
Practical Fabric Eventstream patterns: what belongs in the no-code event processor, what belongs in KQL, and what to fix before events arrive.
6 min read
Fabric Real-Time Analytics After GA: An End-to-End Streaming Design
How Eventstream, KQL databases, Power BI and the Data Activator preview fit together in Fabric as of January 2024, and when to choose something else.
9 min read
Designing a Gold Layer and Semantic Model That Stay in Direct Lake
Design-time rules for Direct Lake in Fabric: size tables to your SKU's guardrails, move logic into Delta, avoid views, and control when models reframe.
8 min read
Laying Out OneLake: Workspaces, Lakehouses and Shortcuts
How I decide OneLake layout in Fabric: workspaces as security boundaries, a lakehouse per layer, a flat Tables folder, and shortcuts where copies hurt.
7 min read
Tracing a Slow Fabric Report Back Through Every Layer
A diagnose-first workflow for slow Fabric reports: check capacity, the visual, DAX, Direct Lake fallback, Delta file layout and Spark, in that order.
8 min read
Fabric Throttling Explained: Reading Carryforward Before Users Do
How Fabric smoothing, carryforward and the four throttling stages work, how to read them in the Capacity Metrics app, and which levers actually help.
7 min read
Fabric's December 2023 Update: What I'd Act On in January
A practitioner's triage of the Microsoft Fabric December 2023 update: %%configure, Git conflict resolution, KQL in OneLake and what to leave alone.
7 min read
Enterprise AI in 2024: Five Bets Against What Is GA on Azure
Five enterprise AI bets for 2024, each checked against what is generally available or still in preview on Azure as of 1 January 2024.
1 min read
Cost Optimization in Microsoft Fabric
Cost optimisation isn't about cutting features — it's about aligning spend with business value. During holiday slowdowns I help teams identify idle…
1 min read
Fabric Capacity Management: Right-Sizing Your Platform
Right-sizing Fabric capacity is a finance-and-engineering problem: choose a capacity SKU that matches peak loading patterns, instrument CU consumption, and…
1 min read
Query Performance Optimization in Microsoft Fabric
Query performance is where UX and cost meet. My practical approach is to measure user-facing latency, inspect query plans, and prioritise optimisations that…
1 min read
V-Order Optimization: Deep Dive into Fabric's Secret Weapon
V-Order feels like a secret weapon because it's a write-time optimisation with outsized read-time benefits. In practice I enable V-Order on wide, heavily…
1 min read
Direct Lake Optimization: Maximizing Power BI Performance
Direct Lake changes the Power BI performance story — but it's not automatic. Over the past months I've seen Direct Lake deliver dramatic improvements when…
1 min read
Fabric Performance Tuning: Optimizing Your Data Platform
Performance tuning in Fabric blends write-time optimisations (v-ordering, partitioning) with query-time strategies (predicate pushdown, materialised views).…
1 min read
Fabric Components: When to Use What
Fabric's component set is broad; the hard part is picking the minimal surface that solves your business need. This guide distils when to use Lakehouse…
1 min read
Lakehouse vs Warehouse in Fabric: Making the Right Choice
Choosing Lakehouse vs Warehouse is an architectural trade-off: for open-format, large-scale analytics and ML, Lakehouse is the better fit; for classic T-SQL…
1 min read
Fabric Architecture Patterns: Designing for Scale
Designing for scale in Fabric is about decomposition and clear contracts between layers. The medallion architecture works well: small, composable…
1 min read
Microsoft Fabric Best Practices for Production
Shipping Fabric into production exposes common anti-patterns — unbounded capacity use, duplicated data copies, and insufficient observability. The best…
1 min read
Fabric Migration Stories: Real-World Experiences
There is no substitute for learning from real migrations: the patterns we reuse (lift-and-shift for bulk data, iterative modernization for dependent…
1 min read
Microsoft Fabric Adoption: Strategies for Success
A practical Fabric adoption roadmap balances fast wins with platform work — start with a high-impact pilot, instrument the value stream, and invest in a…
4 min read
Microsoft Fabric is GA: What We Learned from Six Months of Preview
Microsoft Fabric reached General Availability at Ignite 2023 — November 15-17 in Seattle — and it's the outcome I genuinely thought might take another six…
1 min read
Copilot in Microsoft Fabric: AI-Powered Analytics
Copilot in Microsoft Fabric was announced at Ignite 2023 as the AI-powered natural language layer across the Fabric experience. The capabilities at launch…
1 min read
Microsoft Purview and Fabric: Enterprise Data Governance
Microsoft Purview and Fabric integration is the enterprise governance story that matters most for regulated industries and large organisations where data…
2 min read
Data Lineage in Microsoft Fabric: Tracking Data Flow
Data lineage in Fabric — the ability to trace where a piece of data came from, what transformations it passed through, and what items depend on it — is…
1 min read
Fabric Endorsement: Certifying Trusted Data Products
Fabric endorsement — the Promoted and Certified badge system for Fabric items — is a lightweight but important trust signal in a self-service analytics…
1 min read
Fabric Domains: Organizing Your Data Mesh
Fabric Domains are the tenant-level organisational construct that lets large enterprises apply data mesh principles to their Fabric deployment — grouping…
2 min read
Fabric Governance: Building a Secure Data Platform
Fabric governance is where the platform's SaaS architecture creates both an advantage and a complication for enterprise security teams. The advantage…
2 min read
Microsoft Fabric Licensing and Capacity Planning Guide
Fabric licensing is genuinely different from the Azure pay-as-you-go mental model that most Azure practitioners are familiar with, and understanding the…
2 min read
Microsoft Fabric GA: The Unified Data Platform Revolution
Microsoft Fabric reaching General Availability at Ignite 2023 (November 15-17) is the milestone that transforms the six-month public preview from an…
1 min read
Microsoft Fabric Best Practices: Lessons Learned
Best practices and lessons learned from implementing Microsoft Fabric solutions for enterprise analytics.
1 min read
ML Model Deployment Patterns in Microsoft Fabric
Best practices and patterns for deploying machine learning models in Microsoft Fabric.
1 min read
The PREDICT Function in Microsoft Fabric
Using the PREDICT function to apply machine learning models directly in Microsoft Fabric for seamless predictions.
1 min read
SemPy Library: Python for Semantic Models in Fabric
Deep dive into the SemPy library for working with Power BI semantic models in Microsoft Fabric notebooks.
1 min read
Semantic Link in Microsoft Fabric: Connecting Data and Analytics
Understanding Semantic Link in Microsoft Fabric for seamless integration between Power BI datasets and Spark notebooks.
1 min read
Data Science Workflows in Microsoft Fabric
Building end-to-end data science workflows using Microsoft Fabric's integrated capabilities.
1 min read
AI Integration in Microsoft Fabric
Exploring AI and machine learning capabilities within Microsoft Fabric for intelligent analytics.
1 min read
Fabric Copilot Preview: AI-Assisted Analytics
Exploring Fabric Copilot capabilities for AI-assisted data engineering, analysis, and reporting.
1 min read
Microsoft Fabric Updates: Unified Analytics Platform
Exploring the latest Microsoft Fabric updates and how they transform enterprise data analytics.
1 min read
Experiment Tracking in Fabric: Best Practices and Patterns
Tomorrow we'll explore Azure OpenAI Code Interpreter. MLflow Tracking Experiment Best Practices MLOps Patterns
1 min read
MLflow in Fabric: Experiment Tracking and Model Management
MLflow in Fabric is the managed tracking layer that turns a Spark notebook into a reproducible experiment record. The integration is transparent — you…
1 min read
ML Models in Fabric: Training, Registry, and Deployment
Tomorrow we'll explore MLflow integration in Fabric. Model Management in Fabric MLflow Model Registry Model Deployment Guide
1 min read
Fabric Data Science: ML Workflows in the Modern Data Platform
Data Science in Fabric brings ML experiment tracking, model registration, and batch prediction into the same platform where the training data lives — which…
1 min read
Real-Time Dashboards in Fabric: Live Data Visualization
Tomorrow we'll explore Fabric Data Science capabilities. Real Time Dashboard Documentation KQL Visualizations Dashboard Best Practices
1 min read
Eventstreams in Fabric: Streaming Data Ingestion
Tomorrow we'll explore Real Time Dashboards for visualizing streaming data. Eventstream Documentation Event Hub Integration Streaming Best Practices
1 min read
KQL Databases: Advanced Patterns and Techniques
'{ "SoftDeletePeriod": "90.00:00:00", "Recoverability": "Enabled" }' '{ "DataHotSpan": "7.00:00:00", "IndexHotSpan": "14.00:00:00" }'
1 min read
Fabric Real-Time Analytics: Stream Processing at Scale
'{' ' "SoftDeletePeriod": "30.00:00:00",' ' "Recoverability": "Enabled"' '}'
1 min read
Query Insights in Fabric: Monitoring and Optimization
Tomorrow we'll explore Fabric Real Time Analytics. Query Insights Documentation Performance Tuning Monitoring DMVs
1 min read
Cross-Database Queries in Fabric: Unified Data Access
Tomorrow we'll explore Query Insights for monitoring and optimization. Cross Database Queries OneLake Shortcuts Security Model
1 min read
T-SQL in Fabric: Patterns and Best Practices
If you're coming to Fabric Warehouse from a SQL Server or Azure SQL Database background, the T-SQL compatibility is reassuring — CTEs, window functions…
1 min read
Fabric Warehouse: Cloud-Native SQL Analytics
The Fabric Warehouse is the part of Microsoft Fabric that surprised me most in hands-on testing. I expected it to be the Lakehouse's SQL analytics endpoint…
1 min read
Fabric Data Pipelines: Orchestration Patterns
A Fabric pipeline isn't just a sequence of activities — it's an orchestration layer with conditional branching, parameter passing, loops, and error handling…
1 min read
Dataflow Gen2: Power Query Transformations in Fabric
A month into production-style Fabric testing and Dataflow Gen2 has become my go-to recommendation for teams without Spark expertise who need repeatable data…
1 min read
Copy Activity in Fabric: Data Movement Patterns
The Copy Activity in Fabric Data Factory is the same copy engine as Azure Data Factory — the underlying data movement service optimised for throughput with…
1 min read
Fabric Data Factory: Data Integration Reimagined
Tomorrow we'll dive deeper into Copy Activity patterns. Data Factory in Fabric Data Pipeline Documentation Pipeline Expressions
1 min read
Data Wrangling in Microsoft Fabric: Practical Techniques
Data wrangling — getting messy, inconsistent, real-world data into a reliable clean form — is where most data engineering hours actually go. In Fabric…
1 min read
Fabric Notebooks: Features and Productivity Tips
Fabric Notebooks have some features I didn't notice until I'd been using them for a few weeks and wish someone had pointed out earlier. Inline data…
1 min read
Apache Spark in Microsoft Fabric: Getting Started
Fabric provides a managed Spark experience: Tomorrow we'll dive deeper into Fabric notebooks and their unique features. Spark in Fabric PySpark…
1 min read
Delta Tables in Microsoft Fabric: Deep Dive
The theoretical benefits of Delta Lake — ACID transactions, time travel, schema enforcement — are easy to describe, but the practical payoff becomes clear…
1 min read
Creating Your First Fabric Lakehouse: Step-by-Step Guide
A Lakehouse combines the best of data lakes and data warehouses: Once created, your Lakehouse has two main sections: The Files section is for unstructured…
1 min read
OneLake Explorer: Navigating Your Data Lake in Fabric
OneLake Explorer is a Windows desktop application that lets you browse the contents of your organisation's OneLake the same way you'd navigate files in…
1 min read
Fabric Workspace Setup: Organization and Best Practices
How you structure workspaces in Fabric matters more than most getting-started guides acknowledge. A workspace is the unit of security, capacity binding, and…
2 min read
Microsoft Fabric Trial: What You Get and How to Maximize It
The Fabric trial is one of the more generous evaluation paths Microsoft has offered for a data platform — a 60-day, F64-equivalent capacity for free is a…
2 min read
Getting Started with Microsoft Fabric: A Practical Guide
I've been onboarding several clients onto Fabric over the past few weeks and the first hour is always the same: getting oriented to an interface that's…
2 min read
Microsoft Fabric Public Preview: What You Need to Know
Six weeks into the Fabric public preview and I'm past the demo phase — I'm running actual data engineering workloads and discovering what holds up and what…
1 min read
Row-Level Security in Fabric: Data Access at the Row Level
RLS provides essential data security for multi-tenant and sensitive data scenarios. Tomorrow, I will cover Azure OpenAI function calling patterns.
1 min read
Item Permissions in Fabric: Granular Access Control
Item permissions provide the flexibility to share specific artifacts securely. Tomorrow, I will cover Row-Level Security in Fabric.
1 min read
Fabric Workspace Roles: Managing Access and Collaboration
Proper role management is essential for secure and efficient collaboration. Tomorrow, I will cover Item Permissions in more detail.
1 min read
Microsoft Fabric Security: Protecting Your Data Platform
Security is an ongoing responsibility. Tomorrow, I will cover Workspace Roles in more detail.
1 min read
Fabric Capacity Metrics: Understanding and Optimizing Consumption
Fabric's capacity unit (CU) model is genuinely different from the per-resource billing in Synapse Analytics, and understanding it properly is the difference…
1 min read
Fabric Monitoring: Observability for Your Data Platform
One of my first questions when evaluating any data platform for enterprise use is "how do I see what's actually running?" — and in the early Fabric…
1 min read
Fabric Deployment Pipelines: Promoting Content Across Environments
Deployment Pipelines provide controlled, auditable content promotion across environments. Tomorrow, I will cover Fabric Monitoring.
1 min read
Fabric Git Integration: Version Control for Your Data Platform
Git integration in Fabric is something I've been waiting for since the first preview. Power BI's lack of proper version control has been a persistent…
1 min read
Fabric Mirroring: Real-Time Database Replication
Mirroring simplifies real-time data replication to Fabric. Tomorrow, I will cover Git integration in Fabric.
1 min read
Dataflows Gen2: Low-Code Data Transformation in Fabric
Dataflows Gen2 provide an accessible way to build data transformations. Tomorrow, I will cover Mirroring in Fabric.
1 min read
Fabric Pipelines: Orchestrating Data Workflows
Fabric Pipelines enable robust data orchestration with full control flow capabilities. Tomorrow, I will cover Dataflows Gen2.
1 min read
Fabric Notebooks: Interactive Data Engineering and Science
Notebooks are the primary development environment in Fabric. Tomorrow, I will cover Fabric Pipelines for orchestration.
1 min read
Semantic Models in Microsoft Fabric: Building the Analytics Layer
Semantic models provide the business logic layer that enables self-service analytics. Tomorrow, I will cover Fabric Notebooks.
1 min read
Direct Lake Mode: Power BI Performance Without Import Overhead
Direct Lake is the quiet superpower of Microsoft Fabric, and I think it's underexplained in most of the post-Build coverage. The classic Power BI trade-off…
1 min read
OneLake Shortcuts: Connecting Data Without Copying
Of everything announced at Build 2023, OneLake shortcuts might be the feature I find most architecturally elegant. A shortcut is not a copy, not a linked…
1 min read
Delta Lake in Microsoft Fabric: ACID Transactions for Data Lakes
Delta Lake isn't a bolt-on feature in Microsoft Fabric — it's the foundational table format for everything. Every table you write through a Spark notebook…
1 min read
Microsoft Fabric Lakehouse Architecture: Deep Dive
A week after Build 2023, I'm still unpacking what Microsoft Fabric actually means for how we design data platforms. The Lakehouse is the centrepiece — but…
1 min read
Microsoft Build 2023 Summary: The AI-Powered Future of Development
Build 2023 had two dominant themes: 1. AI Everywhere : Generative AI integrated into every product and platform 2. Platform Unification : Simplifying…
5 min read
Microsoft Fabric: The Biggest Data Launch Since SQL Server
At Microsoft Build 2023 on May 23, Satya Nadella announced Microsoft Fabric with the phrase "perhaps the biggest launch of a data product from Microsoft…
2 min read
Microsoft Fabric Unveiled at Build 2023: The Future of Data Analytics
All in ONE cohesive experience with ONE security model and ONE business model. At the heart of Fabric is OneLake - think of it as "OneDrive for data." It's…
1 min read
Migrating to Microsoft Fabric: Strategies and Best Practices
The fastest approach uses OneLake shortcuts to reference existing data without copying. Run Fabric alongside existing platform during transition.
1 min read
Microsoft Fabric vs Azure Synapse Analytics: A Detailed Comparison
Both Synapse and Fabric are excellent platforms. The choice depends on your specific requirements, existing investments, and organizational priorities.
2 min read
Microsoft Fabric Workspace: Organization and Collaboration
A Fabric workspace is the collaboration boundary in Microsoft Fabric—the container where Fabric items (Lakehouses, Warehouses, Notebooks, Pipelines, KQL…
1 min read
Microsoft Fabric Licensing: Understanding Your Options
Understanding Fabric licensing helps you plan costs and ensure users have appropriate access. Tomorrow, I will cover Fabric Workspace configuration and…
1 min read
Understanding Microsoft Fabric Capacities and SKUs
A Fabric Capacity is a pool of compute resources that powers all Fabric workloads in assigned workspaces. Unlike traditional Azure services where you…
1 min read
Power BI in Microsoft Fabric: Unified Business Intelligence
Power BI in Fabric provides seamless BI capabilities that connect directly to your unified data layer. Tomorrow, I will cover Fabric Capacities and pricing.
1 min read
Synapse Real-Time Analytics in Microsoft Fabric: Streaming at Scale
Real-Time Analytics in Fabric provides powerful streaming capabilities for time-sensitive workloads. Tomorrow, I will cover Power BI integration in Fabric.
1 min read
Synapse Data Warehouse in Microsoft Fabric: Cloud-Native SQL Analytics
The Fabric Data Warehouse provides enterprise SQL analytics without infrastructure management. Tomorrow, I will cover Synapse Real-Time Analytics for…
1 min read
Synapse Data Science in Microsoft Fabric: ML Made Simple
Fabric's Data Science experience integrates seamlessly with the rest of the platform, allowing you to go from raw data in Lakehouse to deployed models in a…
1 min read
Synapse Data Engineering in Microsoft Fabric: Spark at Scale
Synapse Data Engineering in Fabric provides a managed Spark experience that removes infrastructure concerns while delivering the full power of Apache Spark.…
1 min read
Data Factory in Microsoft Fabric: Modern Data Integration
Data Pipelines in Fabric are similar to Azure Data Factory pipelines but with deeper Fabric integration.
1 min read
Building with Fabric Lakehouse: The Best of Data Lakes and Warehouses
The Lakehouse is where most of your Fabric data engineering work will happen. Tomorrow, I will cover Data Factory in Fabric for orchestrating data movement…
1 min read
OneLake: The Foundation of Microsoft Fabric
OneLake is a single, unified, logical data lake for your entire organization. Think of it as "OneDrive for data" - automatically provisioned when you enable…
1 min read
Understanding Microsoft Fabric as a Unified Analytics Platform
This fragmentation creates operational overhead that Fabric aims to eliminate.