1 min read
Building Data Lakehouses: Medallion Architecture Best Practices
Each layer serves a specific purpose in the data refinement pipeline. Bronze captures raw data, silver cleans and conforms it, and gold delivers…
Lakehouse
21 articles
1 min read
Delta Lake Deep Dive: Time Travel and Schema Evolution in Fabric
Every write operation to a Delta table creates a new version. This transaction log enables point-in-time queries and rollback capabilities essential for…
1 min read
Implementing Data Quality Checks in Microsoft Fabric Lakehouses
Poor data quality leads to incorrect business decisions, failed ML models, and eroded trust in analytics platforms. A proactive approach to data validation…
1 min read
Implementing Retrieval-Augmented Generation with Microsoft Fabric
Use Fabric notebooks to create interactive RAG applications that combine data exploration with AI-powered question answering over enterprise datasets.
1 min read
Microsoft Fabric Lakehouse: Medallion Architecture Implementation
Bronze holds raw ingested data. Silver contains cleansed and conformed data. Gold presents business-level aggregates ready for consumption.
1 min read
Medallion Architecture in Microsoft Fabric: Bronze, Silver, Gold Explained
Bronze tables store data exactly as received from source systems, preserving the original format for replayability and debugging.
1 min read
Apache Iceberg on Microsoft Fabric: Open Table Format Integration
Iceberg provides ACID transactions, schema evolution, time travel, and partition evolution for data lakes. Its format-agnostic design works with Spark…
1 min read
Microsoft Fabric Lakehouse: Unified Analytics for the Modern Data Stack
A Lakehouse stores data in open formats like Delta Lake while providing SQL query capabilities, ACID transactions, and schema enforcement. In Microsoft…
1 min read
Lakehouse Federation in Microsoft Fabric
1. Define clear domains : Each lakehouse should own specific data 2. Minimize cross domain joins : Materialize frequently joined data 3. Implement…
2 min read
Microsoft Fabric Data Engineering: Best Practices After 6 Months in Production
Avoid over-partitioning. If partitions have < 1GB of data, consolidate. Use Data Pipelines, Not Just Notebooks
1 min read
Data Platform Evolution: Where We're Heading
Having lived through the shift from monolithic warehouses to lakehouses, the pattern I'm seeing now is convergence: open formats, governance, and…
1 min read
Lakehouse vs Warehouse in Fabric: Making the Right Choice
Choosing Lakehouse vs Warehouse is an architectural trade-off: for open-format, large-scale analytics and ML, Lakehouse is the better fit; for classic T-SQL…
1 min read
Fabric Architecture Patterns: Designing for Scale
Designing for scale in Fabric is about decomposition and clear contracts between layers. The medallion architecture works well: small, composable…
1 min read
Delta Tables in Microsoft Fabric: Deep Dive
The theoretical benefits of Delta Lake — ACID transactions, time travel, schema enforcement — are easy to describe, but the practical payoff becomes clear…
1 min read
Creating Your First Fabric Lakehouse: Step-by-Step Guide
A Lakehouse combines the best of data lakes and data warehouses: Once created, your Lakehouse has two main sections: The Files section is for unstructured…
1 min read
Delta Lake in Microsoft Fabric: ACID Transactions for Data Lakes
Delta Lake isn't a bolt-on feature in Microsoft Fabric — it's the foundational table format for everything. Every table you write through a Spark notebook…
1 min read
Microsoft Fabric Lakehouse Architecture: Deep Dive
A week after Build 2023, I'm still unpacking what Microsoft Fabric actually means for how we design data platforms. The Lakehouse is the centrepiece — but…
1 min read
Building with Fabric Lakehouse: The Best of Data Lakes and Warehouses
The Lakehouse is where most of your Fabric data engineering work will happen. Tomorrow, I will cover Data Factory in Fabric for orchestrating data movement…
1 min read
Data Engineering Patterns That Defined 2022
2022 established these patterns as the foundation of modern data engineering. The lakehouse became the default architecture for new data platforms. Data…
2 min read
Getting Started with Delta Lake on Azure Databricks
Data lakes have a long-standing reputation problem: cheap to fill, painful to trust. Schema drift, half-written files from failed jobs, "is this row a…
1 min read
Databricks Delta Lake: Advanced Patterns
Delta Lake: the foundation of the modern lakehouse.