Thoughts on data engineering, cloud architecture, and technology. 2213 articles and counting.
Showing 24 of 2213 posts — page 1 of 93
I asked a Fabric data agent 24 questions in three wordings, ten times each, on real AEMO data. It scored 82%, but only 39% of wordings were always right.
I gave Opus 5.5 a build to watch and a dev server to start, with and without the Monitor tool. It used Monitor inconsistently, and headless runs hid the answer.
ultrathink now runs at xhigh effort in Claude Code. I ran six exact-answer tasks 96 times from low to max: every level got them right; higher just cost more.
I had Haiku 5.5 audit 30 of my posts twice: once with --json-schema, once asking nicely for JSON. The shape held. The judgement moved, and that's the lesson.
I hid five bugs in a real change and ran /code-review at low, medium, high and xhigh. Medium found all five for 15 cents. Here's what each level adds.
Slides is out of beta. I had Claude Code turn one of my posts into a nine-slide deck, then checked every claim against the source. Here's what held up.
A Claude Code mod to hold destructive Azure CLI commands passed 12 unit tests and stopped nothing live. Here's why, the fix, and the gap no regex closes.
I gave Haiku 5.5 and Opus 5.5 the same five chores in this blog's repo. Both passed every run. Haiku cost 25 times less, and the diffs show what Opus adds.
Every source schema changes eventually. Which changes Delta accepts silently, which it rejects, a contract per layer, and the changes no schema check sees.
Should last year's revenue move when a customer changes segment? Pick an SCD type per column, write a replay-safe Type 2 MERGE, and join facts correctly.
APPROXIMATEDISTINCTCOUNT now runs on Import and Direct Lake models in preview. Decide per measure where about 1.6% error is fine, label it, and test the gain.
Document Intelligence wins on fixed layouts, Content Understanding on variable or multimodal input: split the pipeline and leave stable extractors alone.
Small files come from simple arithmetic, and every reader pays a cost per file. Where they come from, how to measure them, and how to compact them safely.
Fabric's preview data engineering agent runs with the launching engineer's permissions. What to delegate first, which settings to pick, and what stays human.
Microsoft backed Apache Ossie at FabCon with a Power BI converter. It makes metric names portable, not DAX semantics, security rules or performance.
Fabric mirroring for BigQuery is GA, but it moves cost to Google: CDC compute, egress, keyless reseeds and schema ownership decide when to mirror or copy.
Fabric IQ is GA in Copilot Chat, Cowork and MCP. Treat each semantic model as an agent endpoint: RLS, OLS, metadata contracts and default-on access.
DiskANN vector indexes are now GA in Azure SQL and Fabric SQL. When embeddings belong next to your rows, and when Azure AI Search still wins.
A query for one day should not read three years. How partition pruning works, the five ways it silently breaks, and how to pick partition columns.
Foundry routines run an agent on a timer, a schedule or a GitHub or Teams event. When to use them, when to use a workflow, and guardrails for unattended runs.
Foundry's preview egress controls attach ordered destination rules to a hosted agent. Why that limits prompt-injection damage, and how to roll it out safely.
GPT-6 arrived in Foundry ten weeks after GPT-5.6. Pin versions, gate on real queries, shadow traffic and plan retirements before you switch tiers on day one.
The physical plan shows what Spark will actually do: shuffles, join strategy, pruning, Python UDFs. How to read one, and which plans should worry you.
Eventstream can now read Azure Event Hubs with workspace identity. How to move off SAS keys, the RBAC each side needs, and when a service principal still wins.