8 min read
Laying Out OneLake: Workspaces, Lakehouses and Shortcuts
How I decide OneLake layout in Fabric: workspaces as security boundaries, a lakehouse per layer, a flat Tables folder, and shortcuts where copies hurt.
Blog
Thoughts on data engineering, cloud architecture, and technology. 2205 articles and counting.
Showing 24 of 2205 posts — page 38 of 92
7 min read
Tracing a Slow Fabric Report Back Through Every Layer
A diagnose-first workflow for slow Fabric reports: check capacity, the visual, DAX, Direct Lake fallback, Delta file layout and Spark, in that order.
8 min read
Fabric Throttling Explained: Reading Carryforward Before Users Do
How Fabric smoothing, carryforward and the four throttling stages work, how to read them in the Capacity Metrics app, and which levers actually help.
7 min read
Fabric's December 2023 Update: What I'd Act On in January
A practitioner's triage of the Microsoft Fabric December 2023 update: %%configure, Git conflict resolution, KQL in OneLake and what to leave alone.
7 min read
LangChain, AutoGen, CrewAI or Semantic Kernel? Choosing in January 2024
How LangChain 0.1, AutoGen 0.2, CrewAI 0.1 and Semantic Kernel 1.0 differ in maturity, control and Azure OpenAI fit, and which one I'd pick for what.
6 min read
CrewAI 0.1 on Azure OpenAI: Roles, Tasks and Current Limits
A practical look at CrewAI 0.1.x with Azure OpenAI: how agents, tasks and sequential crews work, what's missing today, and when to choose it.
7 min read
AutoGen 0.2 on Azure OpenAI: Termination, Code Execution and Tools
A practical look at AutoGen 0.2 with Azure OpenAI: wiring the config, stopping runaway conversations, sandboxing generated code and registering tools.
7 min read
When to Split One AI Agent into Several: A Supervisor Pattern
When a single GPT-4 agent stops being reliable, split it into specialists. A framework-free supervisor pattern on Azure OpenAI, with trade-offs and limits.
6 min read
Self-Correcting RAG: Grade Retrieval Before You Answer
How to add retrieval grading, query rewriting, and groundedness checks to RAG on Azure OpenAI and Azure AI Search without tripling latency or cost.
6 min read
Structure-Aware Chunking for RAG: Headings, Tables and Breadcrumbs
Chunk documents along their headings, keep tables whole and prefix every chunk with its section path, using Document Intelligence markdown output and Python.
7 min read
A RAG Maturity Model: Diagnosing Where Your System Falls Short
A five-level RAG maturity model for Azure AI Search and Azure OpenAI, covering what each level fixes, what it costs, and when to stop climbing.
7 min read
Governing Custom GPTs in ChatGPT Enterprise Before the Store Opens
How to run internal-only GPTs in ChatGPT Enterprise: sharing controls, what knowledge files really expose, Actions auth, and when to build instead.
7 min read
Capture, Plan, Execute, Refine: My Knowledge Work System
The four-stage personal workflow I use as a knowledge worker, and the tools behind each stage: reMarkable, Drafts, Readwise, Obsidian, To Do and Outlook.
7 min read
Owning the Loop: AI Assistant Architecture on Chat Completions
How to structure a production AI assistant on Azure OpenAI Chat Completions: conversation state, retrieval, tool calling and the orchestration loop you own.
8 min read
Assistants API Runs in Production: Polling, Tool Calls and Expiry
How to drive OpenAI Assistants API runs reliably in January 2024: the run state machine, polling, function calls, expiry, concurrency and cost.
7 min read
Shipping on GPT-4 Turbo Preview in Azure OpenAI: A Production Checklist
GPT-4 Turbo is still a preview model on Azure OpenAI; here's how I'd deploy, budget, retry and test around 1106-preview before real users touch it.
7 min read
Enterprise AI in 2024: Five Bets Against What Is GA on Azure
Five enterprise AI bets for 2024, each checked against what is generally available or still in preview on Azure as of 1 January 2024.
1 min read
Goodbye 2023, Hello 2024: What's Next in AI and Data
2023 closed with more platform and model announcements than any year prior. My central thesis is simple: treat AI as a product — instrument, govern, and…
1 min read
2023 Summary: Key Takeaways for Data and AI Professionals
From enterprise pilots to platform bets, 2023 was the year AI stopped being optional. These are the key takeaways I share with clients when they ask what's…
1 min read
Platform Engineering Trends: Building Internal Data Platforms
Platform engineering is the pragmatic answer to scale: provide golden paths, self-service APIs and guardrails so teams move quickly without repeating…
1 min read
Data Platform Evolution: Where We're Heading
Having lived through the shift from monolithic warehouses to lakehouses, the pattern I'm seeing now is convergence: open formats, governance, and…
1 min read
AI Trends to Watch in 2024
There are a few themes I'm watching closely for 2024 — practical agents, small-model efficiency gains, and tighter integration between data platforms and…
1 min read
2024 Predictions: The Year Ahead in AI and Data
Predictions are inherently risky, but after a year working closely with foundation models and platform teams, a few trends feel probable: larger context…
1 min read
Cost Optimization in Microsoft Fabric
Cost optimisation isn't about cutting features — it's about aligning spend with business value. During holiday slowdowns I help teams identify idle…