8 min read
Synapse Serverless varchar(8000): Fixing Strings for Spark and Power BI
Why Synapse serverless SQL infers strings as varchar(8000), why Spark-created lake tables behave differently, and where Power BI string contracts belong.
Synapse
37 articles
9 min read
Synapse to Fabric Migration: 5 Things I Wish I'd Known
Our Synapse to Fabric migration took twice as long as planned. Five lessons on scope, notebook refactoring, Direct Lake, capacity monitoring and training.
2 min read
Top 10 Azure Data Services Lessons from 2025
After a year of implementing data solutions on Azure, I've compiled the most important lessons that can save you time, money, and headaches. These come from…
1 min read
Azure Synapse Analytics AI: Machine Learning at Scale
Synapse enables AI at data warehouse scale with native ML integration.
1 min read
Synapse Data Engineering in Microsoft Fabric: Spark at Scale
Synapse Data Engineering in Fabric provides a managed Spark experience that removes infrastructure concerns while delivering the full power of Apache Spark.…
1 min read
Cognitive Services in Azure Synapse: Built-in AI Capabilities
Cognitive Services in Synapse brings AI capabilities directly into your data engineering workflows. Process millions of records with sentiment, entities…
1 min read
Azure Synapse Analytics + OpenAI: Intelligent Data Warehousing
Azure Synapse with OpenAI transforms data warehousing from technical SQL expertise to conversational data access. Business users can explore enterprise data…
1 min read
Azure Log Analytics Integration with Synapse
Configure diagnostic settings to export to Data Lake: Use serverless SQL to query exported logs: Complex log analysis: Process logs with Azure Functions:…
1 min read
Synapse Data Explorer: Real-Time Analytics at Scale
// Enable streaming ingestion .alter database SensorAnalytics policy streamingingestion enable
1 min read
Azure Synapse Analytics: Build 2022 Updates
Connect operational data to analytics: Enhanced query capabilities: Use Spark for complex analytics: Time series and log analytics: Build data pipelines:…
1 min read
Data Flows Gen2: Visual Data Transformation at Scale
Data Flows Gen2 democratize data transformation, enabling both developers and data analysts to build scalable ETL solutions.
1 min read
Synapse Spark Pools: Optimization and Best Practices
Properly optimized Spark pools can process petabytes of data efficiently and cost-effectively.
1 min read
Synapse Dedicated SQL Pools: Performance Tuning Guide
These optimizations can dramatically improve dedicated SQL pool performance while reducing resource consumption.
1 min read
Synapse Serverless SQL Performance: Optimization Strategies
Serverless SQL charges based on data processed. Optimization reduces both cost and query time.
1 min read
Azure Synapse Analytics Updates: What's New in 2022
Azure Synapse Analytics in 2022 provides a more unified, performant, and feature-rich analytics platform.
1 min read
Azure Synapse Link for Dataverse: Real-Time Analytics on Business Data
Azure Synapse Link for Dataverse eliminates the traditional barriers between operational and analytical systems. Business users get insights from live data…
2 min read
Workload Management in Azure Synapse
Workload management in Synapse Dedicated SQL Pool is the resource governance system that prevents low-priority ad-hoc queries from consuming all compute and…
2 min read
Result Set Caching for Query Acceleration in Synapse
Result set caching in Synapse Dedicated SQL Pool stores the output of a query in the dedicated pool's storage and returns the cached result for subsequent…
2 min read
High-Performance Data Loading with COPY Command in Synapse
The COPY command replaced PolyBase as the recommended data loading mechanism for Synapse Dedicated SQL Pool because it's simpler to use and performs at the…
2 min read
PolyBase for Data Virtualization in Azure Synapse
PolyBase is the T-SQL feature that makes Synapse Dedicated SQL Pool a data virtualisation engine, not just a data warehouse—it allows you to define external…
2 min read
Azure Synapse Dedicated SQL Pool Best Practices
Synapse Dedicated SQL Pool (formerly Azure SQL Data Warehouse) is the massively parallel processing (MPP) engine for structured, petabyte-scale analytical…
2 min read
Azure Synapse Serverless SQL Pools for Big Data Analytics
Synapse Serverless SQL Pool is the query engine that turned Azure Data Lake into an ad-hoc analytics platform—you write T-SQL, point it at files in ADLS…
3 min read
Microsoft Ignite 2021: Data, AI, and the Intelligent Cloud
Microsoft Ignite Fall 2021 brought a wave of announcements across Azure. While hybrid work and Microsoft Teams dominated the headlines, the data and AI…
1 min read
Getting Started with Apache Spark 3.0 in Azure Synapse Analytics
First, let us create a new Spark pool with version 3.0 in Azure Synapse: One of the most significant improvements in Spark 3.0 is Adaptive Query Execution.…
1 min read
Visual ETL with Azure Synapse Data Flows
Data flows in Synapse run on Spark clusters and offer: Visual drag and drop interface 80+ built in transformations Schema drift handling Data preview and…
1 min read
Azure Synapse Dedicated SQL Pools for Enterprise Data Warehousing
Dedicated SQL pools distribute data across 60 distributions for parallel processing: Control node : Orchestrates queries and manages metadata Compute nodes…
1 min read
Real-Time Analytics with Cosmos DB Analytical Store
The question that sparked this post came from a data engineer on a client project: "can we run Power BI reports against Cosmos DB without destroying the…
2 min read
Big Data Processing with Azure Synapse Spark Pools
Synapse Spark pools are the feature that stops the "should I use Databricks or Synapse?" question from being purely a product choice and makes it an…
8 min read
Synapse Link for Cosmos DB: Is It Time to Retire the Nightly ETL?
When Azure Synapse Link can replace a Cosmos DB export pipeline, the limits to check first, and how to query the analytical store from SQL and Spark.
11 min read
AI Across the Microsoft Cloud: Choosing the Right Layer in 2021
A practitioner's map of AI and ML options across Azure, Power Platform and Microsoft 365 in early 2021, and how to pick the right layer for a problem.
8 min read
Synapse Serverless SQL After GA: Designing a Lake Query Layer
How I'd structure Synapse serverless SQL now it's GA: a user database, credentials, views over the lake, CETAS, and habits that keep the per-TB bill small.
3 min read
Azure Synapse Analytics is Now Generally Available
All of this is unified through Synapse Studio - a single workspace where data engineers, data scientists, and analysts can collaborate.
1 min read
Azure Synapse Dedicated SQL Pools (Preview): Enterprise Data Warehousing
Note: While dedicated SQL pools are mature (evolved from Azure SQL DW), the Synapse unified experience is still in preview.
1 min read
Azure Synapse Serverless SQL (Preview): Query Your Data Lake
A second Synapse-preview note, on what I think is the genuinely new idea in the platform: serverless SQL. Point T-SQL at parquet, CSV, or JSON in your data…
1 min read
Azure Synapse Spark Pools (Preview): Big Data Processing
Spark on Synapse is the same managed Spark story most cloud providers offer now, but it's done with the parts of Azure I already use — same workspace as the…
1 min read
Azure Synapse Link for Cosmos DB: Real-Time Analytics
Synapse Link: analytics without the pipeline complexity.
7 min read
Azure Synapse Analytics in Preview: What's GA and Where to Start
A first-day tour of Azure Synapse Analytics in August 2020: which parts are GA, which are preview, and why I start with SQL on-demand before a SQL pool.