1 min read
Delta Lake Everywhere: The Universal Table Format
Delta Lake provides a reliable, performant foundation for modern data platforms. Its ACID transactions, time travel, and schema evolution capabilities make…
Data Lake
12 articles
1 min read
OneLake Interoperability: Connecting Your Data Ecosystem
1. Use managed identity : Preferred for Azure services 2. Leverage shortcuts : Don't copy data unnecessarily 3. Optimize file sizes : 128MB 1GB for best…
1 min read
OneLake Explorer: Navigating Your Data Lake in Fabric
OneLake Explorer is a Windows desktop application that lets you browse the contents of your organisation's OneLake the same way you'd navigate files in…
1 min read
OneLake: The Foundation of Microsoft Fabric
OneLake is a single, unified, logical data lake for your entire organization. Think of it as "OneDrive for data" - automatically provisioned when you enable…
2 min read
External Tables in Azure Data Explorer
External tables in ADX expose data in Azure Blob Storage or ADLS Gen2 as queryable tables without ingesting the data into ADX—useful when the data is too…
1 min read
Azure Data Lake Storage Gen2: Hierarchical Namespace for Big Data Analytics
Azure Data Lake Storage Gen2 provides the foundation for modern data analytics, combining the best of blob storage scalability with the directory semantics…
2 min read
ADLS Gen2 Access Control with ACLs - Fine-Grained Security
Both work together - a user needs both RBAC permission to access the storage account AND appropriate ACL permissions on the specific path.
2 min read
Azure Data Lake Storage Gen2 - Building Modern Data Lakes
ADLS Gen2 is the storage layer that most enterprise data platforms in Australia are converging on. The combination of blob-scale storage (petabytes, low…
2 min read
Delta Lake Time Travel - Versioned Data Lake Queries
Delta Lake time travel is the feature that makes "oops" survivable. An engineer runs a DELETE with a typo in the WHERE clause. A batch job writes corrupt…
8 min read
Synapse Serverless SQL After GA: Designing a Lake Query Layer
How I'd structure Synapse serverless SQL now it's GA: a user database, credentials, views over the lake, CETAS, and habits that keep the per-TB bill small.
1 min read
Azure Data Lake Storage Gen2: Foundation of Modern Data Platforms
Gen2 is the standard for new data lakes on Azure. ADLS Gen2 is the foundation. Build your data platform on solid ground.
7 min read
Azure Synapse Analytics in Preview: What's GA and Where to Start
A first-day tour of Azure Synapse Analytics in August 2020: which parts are GA, which are preview, and why I start with SQL on-demand before a SQL pool.