7 min read
Slowly Changing Dimensions Without Corrupting Your Facts
Should last year's revenue move when a customer changes segment? Pick an SCD type per column, write a replay-safe Type 2 MERGE, and join facts correctly.
Analytics
78 articles
6 min read
The Grain of a Table: Declare It Before the First Join
Grain is what one row of a table means. Most wrong numbers come from a join or aggregation that quietly changed it. How to declare grain and test it.
9 min read
Microsoft Fabric Reality Check: What Holds Up and What Hurts
An honest look at running Microsoft Fabric on an F64: what OneLake and Direct Lake deliver, how throttling really works, and what the bill looks like.
1 min read
Microsoft Fabric Lakehouse: Unified Analytics for the Modern Data Stack
A Lakehouse stores data in open formats like Delta Lake while providing SQL query capabilities, ACID transactions, and schema enforcement. In Microsoft…
1 min read
Power BI Copilot: Natural Language Analytics for Business Users
Use descriptive table and column names that match business terminology. Add measure descriptions explaining what each calculation represents. Create…
1 min read
Measuring AI ROI: Quantifying AI Business Value
Rigorous ROI measurement ensures AI investments deliver demonstrable business value.
1 min read
Azure Synapse Analytics AI: Machine Learning at Scale
Synapse enables AI at data warehouse scale with native ML integration.
1 min read
Microsoft Fabric AI Integration: Unified Analytics and Intelligence
Microsoft Fabric integrates AI throughout the analytics lifecycle from ingestion to insights.
1 min read
Analytics Modernization: From Legacy BI to AI-Powered Insights
Analytics modernization is a journey, not a destination. Start with clear goals, measure progress, and continuously evolve your analytics capabilities.
1 min read
Data Platform Trends: What Shaped 2024 and What's Coming
The data platform landscape is consolidating around unified, AI-native platforms with strong real-time capabilities. Organizations should align their…
1 min read
Microsoft Fabric Year in Review: From GA to Enterprise Standard
1. Start with a focused use case Don't try to migrate everything at once 2. Embrace lakehouse architecture It's the foundation of Fabric 3. Invest in…
1 min read
AI ROI Analysis: Measuring the Real Value of Enterprise AI
AI ROI is real and measurable, but requires disciplined baseline measurement and honest assessment of both costs and benefits.
1 min read
Microsoft Fabric AI Skills: Natural Language Analytics
AI Skills democratize data access while maintaining governance. Start with well-defined domains and expand based on user feedback.
1 min read
Microsoft Fabric at Ignite 2024: What's New for Data Professionals
The Fabric platform continues to mature into a comprehensive, AI-native analytics solution.
1 min read
HTAP Patterns in Microsoft Fabric: Unifying Transactions and Analytics
HTAP in Microsoft Fabric enables unified transactional and analytical workloads without complex ETL pipelines. Design your schemas with both workloads in mind.
1 min read
Microsoft Fabric Updates: October 2024 Announcements
Microsoft announced the preview of relational databases directly within Fabric, bringing transactional workloads into the unified platform.
1 min read
Usage Reporting in Microsoft Fabric
1. Collect comprehensively : Gather data from all sources 2. Aggregate appropriately : Different audiences need different views 3. Automate distribution :…
1 min read
Cross-Cloud Analytics with Microsoft Fabric
Most enterprises use multiple clouds: 1. Minimize cross cloud joins : Materialize frequently joined data 2. Use shortcuts wisely : Only for data that…
1 min read
Cosmos DB Mirroring to Fabric: Analytical Queries at Scale
Unlike SQL mirroring which uses change tracking, Cosmos DB mirroring leverages the built in analytical store: 1. Go to your Cosmos DB account 2. Navigate to…
1 min read
Real-Time Dashboards in Microsoft Fabric
Real-time dashboards bring your streaming data to life. Today I'm exploring how to build effective real-time visualizations in Fabric.
1 min read
Microsoft Fabric June 2024 Updates: Real-Time Intelligence GA
Real-Time Intelligence is Fabric's answer to streaming analytics, combining the power of Azure Data Explorer with the unified Fabric experience.
1 min read
Dashboards in Databricks: Building Data Applications
Databricks dashboards (including Lakeview) provide powerful visualization capabilities integrated with the lakehouse. This guide covers building effective…
1 min read
Databricks Genie Spaces: Natural Language Data Exploration
Genie Spaces enable non-technical users to explore data using natural language. This guide covers setting up and optimizing Genie Spaces for your organization.
1 min read
Databricks AI/BI: Intelligent Analytics Platform
Databricks AI/BI combines the power of the lakehouse with AI-driven analytics. This guide explores how to leverage AI/BI for intelligent data analysis.
1 min read
Azure Databricks April 2024 Updates: AI and Analytics Convergence
Azure Databricks continues to evolve with powerful AI/BI features. April 2024 brings significant updates that blur the line between data engineering and…
1 min read
Auto-Generated Insights: AI-Driven Data Discovery
Auto-generated insights use AI to automatically discover patterns, anomalies, and trends in your data. This guide covers building insight generation systems.
1 min read
Natural Language Queries for Data: Beyond Simple Q&A
Natural language interfaces for data go beyond simple Q&A to enable complex analytical conversations. This guide explores advanced natural language query…
1 min read
Q&A Improvements in Power BI: Natural Language Analytics
Power BI Q&A enables users to ask questions in natural language and receive instant visualizations. Recent improvements make it more accurate and powerful.
1 min read
Smart Narratives in Power BI: Automated Data Storytelling
Smart Narratives automatically generate text summaries of your data, transforming numbers into stories. This guide covers implementation and customization.
1 min read
Copilot in Power BI: AI-Assisted Report Building
Copilot in Power BI revolutionizes how we build reports and analyze data. This guide explores how to effectively use Copilot for report creation and data…
1 min read
Power BI April 2024 Updates: AI-Powered Analytics
Power BI continues to evolve with AI-driven features that make data analysis more accessible. Here's what's new in April 2024.
1 min read
Data Analysis with Copilot: AI-Powered Insights
Microsoft Fabric's Copilot transforms data analysis by enabling natural language interactions with your data. This guide shows how to leverage Copilot for…
1 min read
Fabric Copilot Improvements: Smarter AI Assistance
Microsoft Fabric Copilot has received significant upgrades, making AI-assisted data analysis more powerful and contextually aware. Here's what's new and how…
1 min read
Microsoft Fabric April 2024 Updates: What's New
Microsoft Fabric continues to evolve as a unified analytics platform. April 2024 brings significant updates across data engineering, science, and business…
4 min read
Microsoft Fabric is GA: What We Learned from Six Months of Preview
Microsoft Fabric reached General Availability at Ignite 2023 — November 15-17 in Seattle — and it's the outcome I genuinely thought might take another six…
1 min read
Copilot in Microsoft Fabric: AI-Powered Analytics
Copilot in Microsoft Fabric was announced at Ignite 2023 as the AI-powered natural language layer across the Fabric experience. The capabilities at launch…
2 min read
Microsoft Fabric GA: The Unified Data Platform Revolution
Microsoft Fabric reaching General Availability at Ignite 2023 (November 15-17) is the milestone that transforms the six-month public preview from an…
1 min read
Microsoft Fabric Best Practices: Lessons Learned
Best practices and lessons learned from implementing Microsoft Fabric solutions for enterprise analytics.
1 min read
Semantic Link in Microsoft Fabric: Connecting Data and Analytics
Understanding Semantic Link in Microsoft Fabric for seamless integration between Power BI datasets and Spark notebooks.
1 min read
Fabric Warehouse: Cloud-Native SQL Analytics
The Fabric Warehouse is the part of Microsoft Fabric that surprised me most in hands-on testing. I expected it to be the Lakehouse's SQL analytics endpoint…
2 min read
Microsoft Fabric Public Preview: What You Need to Know
Six weeks into the Fabric public preview and I'm past the demo phase — I'm running actual data engineering workloads and discovering what holds up and what…
1 min read
Semantic Models in Microsoft Fabric: Building the Analytics Layer
Semantic models provide the business logic layer that enables self-service analytics. Tomorrow, I will cover Fabric Notebooks.
1 min read
Direct Lake Mode: Power BI Performance Without Import Overhead
Direct Lake is the quiet superpower of Microsoft Fabric, and I think it's underexplained in most of the post-Build coverage. The classic Power BI trade-off…
5 min read
Microsoft Fabric: The Biggest Data Launch Since SQL Server
At Microsoft Build 2023 on May 23, Satya Nadella announced Microsoft Fabric with the phrase "perhaps the biggest launch of a data product from Microsoft…
1 min read
Synapse Data Warehouse in Microsoft Fabric: Cloud-Native SQL Analytics
The Fabric Data Warehouse provides enterprise SQL analytics without infrastructure management. Tomorrow, I will cover Synapse Real-Time Analytics for…
1 min read
Understanding Microsoft Fabric as a Unified Analytics Platform
This fragmentation creates operational overhead that Fabric aims to eliminate.
1 min read
Building AI Data Analysis Assistants
AI data analysis assistants make insights accessible to everyone, regardless of technical expertise. The key is combining natural language understanding…
1 min read
GPT-4 for Data Analysis: Practical Applications
GPT-4 transforms data analysis from technical SQL writing to conversational exploration. The ability to interpret results and generate insights makes data…
1 min read
Data Platform Evolution in 2022: Trends and Technologies
The convergence of data lakes and data warehouses became mainstream: Streaming data processing became more accessible: Organizations adopted data mesh…
1 min read
Building Real-Time Dashboards with Citus
Combining Citus's distributed query power with rollup tables and caching enables dashboards that handle millions of events per second.
1 min read
Real-Time Analytics on Azure: Building Live Dashboards
Azure's real time analytics stack enables instant insights from millions of events per second, powering live dashboards and alerts.
2 min read
Databricks SQL: Analytics Warehousing on the Lakehouse
Databricks SQL offers: SQL Endpoints : Serverless compute for SQL queries Query Editor : Web based SQL IDE Dashboards : Built in visualization and reporting…
1 min read
Azure Synapse Analytics Updates: What's New in 2022
Azure Synapse Analytics in 2022 provides a more unified, performant, and feature-rich analytics platform.
6 min read
Power BI Smart Narratives vs DAX Text: Choosing How Reports Talk
When the Power BI smart narrative visual is enough, when a hand-written DAX text measure is safer, and the limitations that decide it before you publish.
8 min read
Power BI Automatic Aggregations (Preview): When to Let the Service Decide
How Power BI automatic aggregations train on a DirectQuery dataset's query log, what the preview requires, and when hand-built aggregations still win.
8 min read
Power BI Hybrid Tables: A Live DirectQuery Partition on Import
How Power BI hybrid tables add a DirectQuery partition to an incremental refresh policy, what the preview requires, and when Import alone is the better call.
8 min read
DirectQuery for Power BI Datasets: Extending a Certified Model
How DirectQuery for Power BI datasets lets teams extend a certified model with their own data, what the preview limits are, and when not to chain.
8 min read
Power BI Composite Models: Choosing Import, DirectQuery or Dual
How to set storage modes table by table in a Power BI composite model, when Dual earns its place, and how aggregations keep DirectQuery facts fast.
8 min read
Power BI in Early 2022: What's GA, What's Preview, What to Adopt
A status check on Power BI's newer modelling, performance and ALM features as of January 2022, with my view on which to adopt now and which to pilot.
1 min read
Azure Synapse Link for Dataverse: Real-Time Analytics on Business Data
Azure Synapse Link for Dataverse eliminates the traditional barriers between operational and analytical systems. Business users get insights from live data…
1 min read
Power BI Deployment Pipelines: CI/CD for Business Intelligence
Power BI Deployment Pipelines transform how organizations manage their analytics lifecycle. By bringing CI/CD practices to BI, teams can deliver insights…
1 min read
Power BI Premium Capacities: Scaling Enterprise Analytics
Premium capacities offer: Dedicated compute : No sharing with other tenants Larger datasets : Up to 400GB per dataset Paginated reports : Enterprise…
2 min read
Azure Data Explorer Integration for Advanced Analytics
Azure Data Explorer is the analytics service that addresses a specific limitation of Log Analytics: KQL query performance on large historical datasets…
1 min read
Azure Data Lake Storage Gen2: Hierarchical Namespace for Big Data Analytics
Azure Data Lake Storage Gen2 provides the foundation for modern data analytics, combining the best of blob storage scalability with the directory semantics…
2 min read
Introducing Power BI Goals for OKR and KPI Tracking
Power BI Goals arrived in 2021 as Microsoft's answer to OKR tracking inside the BI platform, and I think it was better conceived than it might sound. The…
1 min read
Power BI Premium Gen2 - Enhanced Performance and New Capabilities
Premium Gen2 introduces several architectural improvements: Autoscale compute resources based on demand No need for capacity management or monitoring v…
1 min read
Databricks SQL Analytics is Now Generally Available
Databricks SQL Analytics provides: Native SQL interface to query Delta Lake tables SQL Endpoints with auto scaling compute Built in visualization and…
2 min read
Spark SQL Fundamentals and Best Practices
Spark SQL is where I've seen the highest productivity for data engineers who already know SQL but are new to distributed processing. You write familiar SQL…
2 min read
Power BI Premium Per User - Enterprise Features for Everyone
Power BI Premium Per User launched as the answer to "my team needs paginated reports, deployment pipelines, and large dataset support but we don't have a…
1 min read
Azure Synapse Dedicated SQL Pools for Enterprise Data Warehousing
Dedicated SQL pools distribute data across 60 distributions for parallel processing: Control node : Orchestrates queries and manages metadata Compute nodes…
1 min read
Real-Time Analytics with Cosmos DB Analytical Store
The question that sparked this post came from a data engineer on a client project: "can we run Power BI reports against Cosmos DB without destroying the…
2 min read
Power BI Dataflows - Best Practices for Enterprise Data Preparation
Dataflows are collections of entities (tables) that are created and managed in workspaces in the Power BI service. They allow you to: Connect to various…
8 min read
Power BI Incremental Refresh in Production: Folding, Partitions, XMLA
What breaks once Power BI incremental refresh is live: query folding, change detection, republishing, and managing partitions through the XMLA endpoint.
3 min read
Azure Synapse Analytics is Now Generally Available
All of this is unified through Synapse Studio - a single workspace where data engineers, data scientists, and analysts can collaborate.
1 min read
Azure Time Series Insights: IoT Analytics at Scale
A general-purpose database can store time-stamped data. It just gets sad about it once you're past a few hundred million rows and someone wants a five-year…
1 min read
Power BI Incremental Refresh: Handle Large Datasets
Right-click table → Incremental refresh Incremental refresh transforms multi-hour refreshes into minutes.
1 min read
Azure Synapse Link for Cosmos DB: Real-Time Analytics
Synapse Link: analytics without the pipeline complexity.