7 min read
Schema Evolution: Surviving the Day Upstream Renames a Column
Every source schema changes eventually. Which changes Delta accepts silently, which it rejects, a contract per layer, and the changes no schema check sees.
Microsoft MVP
Data Platform & Fabric
Michael John Peña
Data & AI Director
Technologist specializing in Microsoft Fabric, Azure data platforms, and cloud architecture. Microsoft MVP and certified trainer helping organizations unlock the power of their data.
10+
Years Experience
MVP
Microsoft MVP
MCT
Certified Trainer
scroll
Expertise
What I Do
Specialized in the Microsoft data ecosystem. From Fabric lakehouses to real-time analytics, I help teams build modern data platforms.
Microsoft Fabric
End-to-end analytics platform expertise including Lakehouses, Data Factory, Real-Time Intelligence, and OneLake architecture.
Azure Data Platform
Designing and implementing enterprise data solutions on Azure including Synapse, Databricks, Data Factory, and Cosmos DB.
AI & Analytics
Integrating AI/ML capabilities with Azure OpenAI, building analytics solutions with Power BI, and implementing MLOps best practices.
Credentials
Microsoft Certifications
Microsoft MVP
- Microsoft Most Valuable Professional (MVP)
Microsoft Certified Trainer
- Microsoft Certified Trainer (MCT)
Expert
- Azure Solutions Architect Expert
- Azure DevOps Engineer Expert
Associate
- Fabric Analytics Engineer
- Fabric Data Engineer
- Azure Data Engineer Associate
- Azure AI Engineer Associate
- Azure Administrator Associate
- Power BI Data Analyst Associate
Tech Stack
Tools I Work With
Microsoft Fabric Platform
Azure Synapse Analytics Analytics
Azure Databricks Data
Azure OpenAI AI
Microsoft Foundry Platform
Claude Anthropic AI
Power BI Analytics
Python Language
Apache Spark Processing
.NET Framework
C# Language
Azure Cosmos DB Database
Azure Kubernetes Infra
Latest Articles
From the Blog
Thoughts on data engineering, Azure, and technology
7 min read
Slowly Changing Dimensions Without Corrupting Your Facts
Should last year's revenue move when a customer changes segment? Pick an SCD type per column, write a replay-safe Type 2 MERGE, and join facts correctly.
6 min read
The Small Files Problem: How Tables End Up With Two Million Files
Small files come from simple arithmetic, and every reader pays a cost per file. Where they come from, how to measure them, and how to compact them safely.
7 min read
Partition Pruning: The Fastest Data Is the Data You Never Read
A query for one day should not read three years. How partition pruning works, the five ways it silently breaks, and how to pick partition columns.
7 min read
Reading a Spark Query Plan: Ten Minutes That Explain a Two-Hour Job
The physical plan shows what Spark will actually do: shuffles, join strategy, pruning, Python UDFs. How to read one, and which plans should worry you.
7 min read
Data Skew: Why 199 Tasks Finish in Seconds and One Takes an Hour
One hot key can make one task do a third of a job, and more nodes will not help. How to spot skew in the Spark UI, find the keys, and fix it properly.
Get in Touch
Let's Build Something Together
Looking for Fabric expertise or Azure data architecture guidance? I'm available for consulting, training, and speaking engagements.