15 articles
A shortcut splices two independently governed permission domains together. Treating it as an access-control and identity problem: two-layer permissions, pass-through vs delegated auth, the Direct Lake / SQL identity-passthrough exception, and where enforcement silently changes hands.
Each layer serves a specific purpose in the data refinement pipeline. Bronze captures raw data, silver cleans and conforms it, and gold delivers…
Implement federated governance that balances domain autonomy with enterprise standards for AI-ready data products.
Data mesh rests on four pillars: domain ownership, data as a product, self-serve data platform, and federated computational governance.
Data mesh is a journey, not a destination. Start with a few high-value domains, prove the model, then expand. The technology enables data mesh, but success…
Unifying Transactional and Analytical Workloads in Fabric
1. Use shared storage : ADLS as the common data layer 2. Standardize on Delta Lake : Compatible format for both 3. Optimize for readers : Both platforms…
Most enterprises use multiple clouds: 1. Minimize cross cloud joins : Materialize frequently joined data 2. Use shortcuts wisely : Only for data that…
1. Define clear domains : Each lakehouse should own specific data 2. Minimize cross domain joins : Materialize frequently joined data 3. Implement…
Domain-Driven Design (DDD) principles apply to data architecture just as they do to software. Today I'm exploring how to design data systems around business…
Fabric Domains are the tenant-level organisational construct that lets large enterprises apply data mesh principles to their Fabric deployment — grouping…
A week after Build 2023, I'm still unpacking what Microsoft Fabric actually means for how we design data platforms. The Lakehouse is the centrepiece — but…
OneLake is a single, unified, logical data lake for your entire organization. Think of it as "OneDrive for data" - automatically provisioned when you enable…
Data mesh isn't a technology choice - it's an organizational paradigm shift. 2021 was about understanding it; 2022 will be about implementing it.
Data engineering in 2021 matured from a support function to a strategic capability. The tools improved, patterns solidified, and the role gained the…