9 min read
Synapse to Fabric Migration: 5 Things I Wish I'd Known
Our Synapse to Fabric migration took twice as long as planned. Five lessons on scope, notebook refactoring, Direct Lake, capacity monitoring and training.
Data
54 articles
9 min read
Databricks or Fabric in 2026: Choosing Your Primary Platform
A practical decision guide for picking Azure Databricks or Microsoft Fabric as your primary data platform in 2026, and when running both is worth the cost.
2 min read
Top 10 Azure Data Services Lessons from 2025
After a year of implementing data solutions on Azure, I've compiled the most important lessons that can save you time, money, and headaches. These come from…
1 min read
Data Lineage: Building Modern AI Systems
Understanding data lineage is essential for production AI systems. Here's what you need to know.
1 min read
Data Governance: Building Modern AI Systems
Understanding data governance is essential for production AI systems. Here's what you need to know.
1 min read
Data Lakehouse Ai: Building Modern AI Systems
Understanding data lakehouse ai is essential for production AI systems. Here's what you need to know.
1 min read
Table Extraction with AI: From Images to Structured Data
AI-powered table extraction handles complex layouts and normalizes data automatically.
1 min read
Semantic Layer for AI: Building Business-Ready Data Interfaces
A semantic layer enables AI to understand business terminology and generate accurate queries.
1 min read
2025 Predictions: What's Next for Data and AI
The future is bright for data and AI professionals who stay current and adapt to these changes.
1 min read
Analytics Modernization: From Legacy BI to AI-Powered Insights
Analytics modernization is a journey, not a destination. Start with clear goals, measure progress, and continuously evolve your analytics capabilities.
2 min read
Microsoft Ignite 2024 Recap: Key Announcements for Data and AI
This unification simplifies the developer experience and provides a clear path from experimentation to production.
1 min read
Feature Engineering in Databricks: From Raw Data to ML Features
Feature engineering transforms raw data into meaningful inputs for machine learning models. Databricks provides powerful tools for building and managing…
1 min read
Databricks AI/BI: Intelligent Analytics Platform
Databricks AI/BI combines the power of the lakehouse with AI-driven analytics. This guide explores how to leverage AI/BI for intelligent data analysis.
1 min read
Auto-Generated Insights: AI-Driven Data Discovery
Auto-generated insights use AI to automatically discover patterns, anomalies, and trends in your data. This guide covers building insight generation systems.
1 min read
Natural Language Queries for Data: Beyond Simple Q&A
Natural language interfaces for data go beyond simple Q&A to enable complex analytical conversations. This guide explores advanced natural language query…
1 min read
Fabric REST APIs: Programmatic Control of Your Analytics Platform
Microsoft Fabric's REST APIs enable automation and integration with your existing tools. This guide covers the key APIs and common use cases.
1 min read
Fabric Git Integration: Version Control for Analytics
Microsoft Fabric's Git integration brings modern version control practices to analytics. This guide covers setting up and using Git with Fabric workspaces.
1 min read
Automated Report Generation with AI
AI can transform raw data into polished, narrative reports. This guide covers building automated report generation systems that combine data analysis with…
1 min read
Natural Language to SQL: Building Intelligent Query Interfaces
Natural language to SQL (NL2SQL) transforms how users interact with databases. This guide covers implementation strategies, from simple approaches to…
1 min read
Fabric Copilot Improvements: Smarter AI Assistance
Microsoft Fabric Copilot has received significant upgrades, making AI-assisted data analysis more powerful and contextually aware. Here's what's new and how…
1 min read
Microsoft Fabric April 2024 Updates: What's New
Microsoft Fabric continues to evolve as a unified analytics platform. April 2024 brings significant updates across data engineering, science, and business…
1 min read
Goodbye 2023, Hello 2024: What's Next in AI and Data
2023 closed with more platform and model announcements than any year prior. My central thesis is simple: treat AI as a product — instrument, govern, and…
1 min read
2023 Summary: Key Takeaways for Data and AI Professionals
From enterprise pilots to platform bets, 2023 was the year AI stopped being optional. These are the key takeaways I share with clients when they ask what's…
1 min read
2024 Predictions: The Year Ahead in AI and Data
Predictions are inherently risky, but after a year working closely with foundation models and platform teams, a few trends feel probable: larger context…
1 min read
LLM-Powered SQL Generation: Patterns and Practices
LLM-powered SQL generation democratizes data access. With proper validation and safety measures, it enables anyone to query databases using natural language.
2 min read
2023 Predictions: Azure, AI, and Data
Closing out 2022 from Australia on New Year's Eve, the technical predictions for 2023 feel more consequential than they ever have—because November 30…
2 min read
Thanksgiving Reflections: A Year of Azure Data and AI Progress
Enterprise features like private endpoints, managed identity, and content filtering make these models production-ready.
1 min read
Knowledge Store in Azure Cognitive Search
Knowledge Store enables rich analytics and knowledge graph construction from AI-enriched content.
1 min read
Temporal Joins: Correlating Events Across Time
Temporal joins and windows enable sophisticated event correlation, essential for IoT analytics, fraud detection, and user behavior analysis.
1 min read
Reference Data Joins in Stream Analytics
Static : Loaded once at job start Periodic : Refreshes at intervals Dynamic : SQL queries for latest data Reference data joins combine streaming data with…
1 min read
Azure Stream Analytics Updates: No-Code Editor and More
The new visual editor allows building streaming pipelines without SQL: 1. Drag and drop input/output connections 2. Visual transformations and aggregations…
1 min read
Dataverse Elastic Tables: High-Volume Data Storage
Elastic tables are ideal for: IoT sensor data Application telemetry Audit logs High frequency transactions Time series data Define an elastic table schema:…
1 min read
Dataverse Virtual Tables: Integrating External Data
Virtual tables provide: Real time access to external data Native Dataverse experience Security role support Power Platform integration Implement a custom…
4 min read
Azure Data and AI: March 2022 Recap and What's Ahead
March 2022 was a productive month for the Azure data and AI ecosystem: the Azure OpenAI Service access expansion brought GPT-3 and Codex to more enterprise…
8 min read
Power BI in Early 2022: What's GA, What's Preview, What to Adopt
A status check on Power BI's newer modelling, performance and ALM features as of January 2022, with my view on which to adopt now and which to pilot.
1 min read
Azure Arc-Enabled Data Services: Run Azure Data Anywhere
Arc-enabled data services bring cloud-native database capabilities to any environment. Whether you need SQL Server or PostgreSQL, you can run managed…
1 min read
Azure Synapse Link for Dataverse: Real-Time Analytics on Business Data
Azure Synapse Link for Dataverse eliminates the traditional barriers between operational and analytical systems. Business users get insights from live data…
1 min read
Power BI Premium Capacities: Scaling Enterprise Analytics
Premium capacities offer: Dedicated compute : No sharing with other tenants Larger datasets : Up to 400GB per dataset Paginated reports : Enterprise…
1 min read
Dataverse for Teams: Building Data-Driven Apps Inside Microsoft Teams
Dataverse for Teams democratizes data platform capabilities. Teams users can now build real applications with proper data management without needing IT…
3 min read
Microsoft Ignite 2021: Data, AI, and the Intelligent Cloud
Microsoft Ignite Fall 2021 brought a wave of announcements across Azure. While hybrid work and Microsoft Teams dominated the headlines, the data and AI…
1 min read
Offline Data Transfer with Azure Data Box
Azure offers several Data Box options: Data Box Disk : Up to 40 TB per order (8 TB per disk) Data Box : 100 TB storage capacity Data Box Heavy : Up to 1 PB…
1 min read
Secure Data Sharing with Azure Data Share
Set up the data sharing infrastructure: Configure datasets to be shared: Invite consumers to access shared data: Set up automated data synchronization: On…
1 min read
Understanding Data Lineage with Azure Purview
Create and configure an Azure Purview account: Register various data sources for lineage tracking: Set up scans to discover and catalog data: Azure Data…
1 min read
Power BI Premium Gen2 - Enhanced Performance and New Capabilities
Premium Gen2 introduces several architectural improvements: Autoscale compute resources based on demand No need for capacity management or monitoring v…
1 min read
Databricks SQL Analytics is Now Generally Available
Databricks SQL Analytics provides: Native SQL interface to query Delta Lake tables SQL Endpoints with auto scaling compute Built in visualization and…
1 min read
Mastering Delta Lake MERGE Operations for Upserts and SCD
The MERGE statement combines INSERT, UPDATE, and DELETE operations in a single atomic transaction: Handle complex business logic with conditional updates:…
1 min read
Getting Started with Apache Spark 3.0 in Azure Synapse Analytics
First, let us create a new Spark pool with version 3.0 in Azure Synapse: One of the most significant improvements in Spark 3.0 is Adaptive Query Execution.…
1 min read
InfluxDB on Azure: Deploying Time-Series Infrastructure
For development or small workloads: For production workloads, use AKS with persistent volumes:
2 min read
TimescaleDB on Azure: Time-Series Data at Scale
TimescaleDB is the time-series database I recommend to teams that are already comfortable with PostgreSQL and don't want to learn a new query language or…
2 min read
Cosmos DB Integrated Cache: Reducing RU Costs at Scale
The Cosmos DB integrated cache is the feature I've been waiting for since the first time I watched a read-heavy application burn through its RU budget…
3 min read
Microsoft Graph Data Connect: Bulk Access to Microsoft 365 Data
Graph Data Connect solves this by delivering bulk extracts of Microsoft 365 data directly to Azure Data Factory.
2 min read
Power BI Dataflows - Best Practices for Enterprise Data Preparation
Dataflows are collections of entities (tables) that are created and managed in workspaces in the Power BI service. They allow you to: Connect to various…
2 min read
Azure Database for PostgreSQL Flexible Server - A Deep Dive
Flexible Server is the deployment option I now default to when a client asks "which PostgreSQL on Azure should I use?" The original Single Server mode…
1 min read
Power BI Incremental Refresh: Handle Large Datasets
Right-click table → Incremental refresh Incremental refresh transforms multi-hour refreshes into minutes.