Azure AI Foundry: What It Is and Why It Matters
A unified platform for building, deploying, and managing AI applications and agents at enterprise scale.
1046 articles
A unified platform for building, deploying, and managing AI applications and agents at enterprise scale.
That's a missed opportunity. Kusto Query Language. Developed by Microsoft. Used across Azure Monitor, Azure Data Explorer, Microsoft Sentinel, and now…
When I tell clients "everything in Fabric uses Delta Lake format," the room divides. Data engineers nod. Everyone else says "what?"
When you send the same prompt prefix repeatedly—system instructions, context documents, examples—the model recomputes them every time. Prompt caching stores…
It's Microsoft's opinionated framework for building production AI agents. Not just chatbots. Agents that reason, plan, use tools, and collaborate with other…
It's Microsoft's opinionated framework for building production AI agents. Not just chatbots. Agents that reason, plan, use tools, and collaborate with other…
This is how most AI deployments start. Here's how to fix it. Traditional apps: request comes in, response goes out. Monitor latency, errors, throughput. Done.
It worked, but managing five services with five billing models and five sets of credentials was painful.
After building production systems on both, here's the honest comparison. Enterprise compliance. Your data stays in your Azure tenant. No data sent to…
This works for demos. It fails in production. Chunking matters more than you think. Random 500-token chunks lose context. A sentence about "the system"…
Most prompt injection attempts never get to your prompt if you filter input properly. Use managed identities. Always.
Spent 3 years with Azure DevOps. Last year switched to GitHub. Here's my honest comparison.
We migrated from Synapse to Fabric. It took twice as long as expected. Here's what I wish someone had told me.
Started with F64 because "enterprise." Spent money we didn't need to. Lesson: Start smaller. F32 was enough. Can always scale up.
Started at $0.08 per query. Too high for our user volume. Switched simple queries from GPT-4o to GPT-4o-mini.
Clients ask me this weekly: "Should I use Databricks or Fabric?" My answer: It depends. Let me explain.
I've deployed vector databases for RAG systems across 5 different projects. Each project chose a different solution. Here's what I actually learned.
Vector search alone misses exact matches. Keyword search alone misses semantic similarity. Combine them.
OneLake is brilliant. One storage layer for everything. No more copying data between Synapse, Data Lake, and Power BI. It just works.
Let me show you where the costs hide. Simple math, right? Wrong. Every chat conversation includes the entire history. That "helpful" feature where the AI…
The tech itself isn't revolutionary—it's the orchestration that's interesting. Code review automation. Agents can catch obvious issues, suggest…
A fun project that combines cloud, IoT, and holiday spirit. Merry Christmas Eve!
Real-time dashboards are now straightforward with Azure SignalR. The serverless model means you only pay for messages, making it cost-effective for most…
Start the new year with optimized cloud spend. These changes compound over time into significant savings.
Zero-trust isn't optional for AI workloads handling sensitive data. Implement these patterns from day one to avoid costly retrofitting.
Reliable messaging is the backbone of microservices architecture. Azure Service Bus provides enterprise-grade messaging, but using it effectively requires…
Fine-tuning is powerful but expensive. Validate the ROI before investing in training infrastructure.
Choose Container Apps when: You want managed infrastructure Your team lacks Kubernetes expertise You have event driven or HTTP workloads You need rapid time…
Observability is essential for AI applications. Without it, you're flying blind on costs, performance, and quality. Implement these patterns early and…
Event-driven architecture requires careful design but delivers superior scalability and resilience for distributed systems.
AI security is not optional. Implement these measures before going to production, and review them quarterly as threats evolve.
Choosing the right vector database is crucial for RAG applications. After implementing production systems with all three major options in 2025, here's my…
Kubernetes cost optimization is continuous. Review these metrics weekly and adjust based on actual usage patterns.
Results: Cold start reduced from 800ms to 150ms. These optimizations combined typically reduce Azure compute costs by 30-50% while improving response times.…
After a year of implementing data solutions on Azure, I've compiled the most important lessons that can save you time, money, and headaches. These come from…
One of the most common questions I receive from enterprise clients is whether to use Azure OpenAI Service or OpenAI's API directly. After working with both…
Many organizations run GPU workloads at 30-40% utilization, paying for idle compute. Understanding workload patterns and implementing optimization…
Many organizations struggle with the gap between experimentation and production. Models that work in notebooks often fail in real-world scenarios due to…
Use this checklist to ensure your organization is ready to quickly evaluate and act on the AI innovations announced at Microsoft Ignite 2025.
The Azure AI platform has been evolving rapidly, and Ignite typically showcases significant platform updates. This year, expect announcements around…
Table extraction transforms static reports into queryable data. Combine extracted tables with LLM analysis to answer questions about financial statements…
Azure offers three custom model approaches: template models for fixed layouts, neural models for varied layouts, and composed models that combine multiple…
First, provision an Azure AI Document Intelligence resource and configure your client. The prebuilt invoice model recognizes vendor details, line items…
Ignite remains the premier event for understanding Microsoft's enterprise AI direction. Planning your learning path around expected announcements helps…
Understanding azure functions ai is essential for production AI systems. Here's what you need to know.
Understanding azure event grid ai is essential for production AI systems. Here's what you need to know.
Strategic use of reserved capacity can reduce AI costs by 30-50% for predictable workloads.
MCP standardizes how AI systems connect to external tools and data sources.
Azure AI Foundry simplifies the path from agent prototype to production deployment.
The new orchestration engine supports complex multi-agent workflows with built-in state management.
Synapse enables AI at data warehouse scale with native ML integration.
Document Intelligence automates data extraction from complex business documents.
Cosmos DB vector search brings global scale and multi-model capabilities to AI applications.
Azure AI Search combines the best of vector, keyword, and semantic search in one service.
Fine-tuning is a powerful tool when used appropriately, but prompt engineering often achieves similar results faster.
Balance quality, speed, and cost based on your specific retrieval needs.
NPUs becoming standard in new PCs Phi 3 and similar SLMs enabling local inference Windows AI features expanding Vector + keyword + sparse becoming standard…
Build copilots that understand context, use tools effectively, and provide clear explanations.
Fabric + AI creates a powerful platform for intelligent analytics. Leverage Spark's scale with AI's intelligence.
AI-powered pipelines transform raw data into intelligent, enriched datasets. Design for both batch and streaming scenarios.
Event-driven AI enables intelligent, automated responses to business events. Start with high-value events where AI can add immediate value.
Streaming inference brings AI insights to real-time data. Design for throughput, latency, and reliability from the start.
Real-time AI requires discipline around latency. Design for the worst case and optimize for the common case.
Audio AI enables natural voice interactions and automated content processing. Choose the right tool for your latency and accuracy requirements.
VLMs unlock powerful visual understanding capabilities. Deploy them thoughtfully with proper optimization and error handling.
Multimodal RAG unlocks knowledge trapped in visual formats. Start with document-heavy use cases where diagrams and charts carry critical information.
Hybrid search continues to outperform pure vector or keyword approaches. Invest in tuning your fusion strategy for your specific domain.
Vector databases in 2025 are more capable, efficient, and integrated than ever. Choose based on your specific requirements for scale, latency, and…
Stay current with Azure AI updates to leverage the latest capabilities for your data and AI applications.
Efficient inference is crucial for production AI. Apply these techniques systematically and measure the impact at each step.
Phi models represent the future of practical AI - powerful enough for real tasks, small enough to deploy anywhere. Start with Phi-3-mini for most use cases…
Build 2025 promises to be significant for data and AI professionals. Stay tuned for the actual announcements and be ready to experiment with new capabilities.
Fabric + AI creates a powerful combination for intelligent analytics. Start with simple enrichments and build toward complex AI-powered workflows.
AI transforms data pipelines from rigid rule-based systems to adaptive, intelligent processes. Start with high-value, error-prone steps and expand from there.
Event-driven AI enables responsive, intelligent systems. Start with simple event handlers and evolve to complex orchestration as needed.
Real-time AI requires careful architecture to balance latency, cost, and quality. Start with simple use cases and optimize based on actual performance data.
Audio AI adds a new dimension to data applications. Start with transcription use cases and expand to voice interfaces as your needs evolve.
Vision-language models transform how we interact with visual data. Start with simple use cases like dashboard analysis and expand to more complex document…
Multimodal RAG opens up new possibilities for enterprise knowledge systems. Start with your most valuable visual content and expand from there.
Hybrid search delivers better results than either approach alone. Implement it early in your RAG pipeline and tune the weights based on your specific use case.
Best for: Enterprise search, RAG applications with complex filtering Best for: Global applications, multi-model data, transactional + vector workloads
Azure OpenAI provides enterprise capabilities, but you need to build robust infrastructure around it for production use.
AI-assisted testing catches more bugs earlier. Combine generated tests with manual review to ensure comprehensive coverage.
AI transforms data quality from reactive checking to proactive management. Start with profiling and anomaly detection, then expand to automated rule…
AI accelerates dbt development but doesn't replace data engineering expertise. Use it to handle boilerplate and documentation while focusing your expertise…
AI-assisted data modeling accelerates the initial design phase but doesn't replace expertise. Use it to generate options quickly, then apply your domain…
Invest in GraphRAG when your use case demands deeper understanding beyond surface-level text matching.
RAG 2.0 is about precision and reliability. Invest in retrieval quality, and your generation quality will follow.
Query: "What's the average order value by customer segment?" SQL: """ Effective prompt engineering is about clear communication. The better you describe…
These patterns form the building blocks of production AI applications. Combine them based on your specific requirements, and always include proper error…
Azure AI Foundry provides the foundation for enterprise AI applications. Start with simple use cases, measure results, and expand from there.
Data mesh is a journey, not a destination. Start with a few high-value domains, prove the model, then expand. The technology enables data mesh, but success…
Real-time analytics in Fabric provides a powerful, integrated solution for streaming data. Start with simple patterns and evolve to more complex scenarios…
Copilot in Fabric is a productivity multiplier. Use it as a starting point, then refine and validate the output. The combination of AI assistance and human…
Microsoft's vision for Fabric in 2025 centers on: 1. AI native analytics Copilot everywhere 2. Real time everything Sub second latency 3. Governance by…
Start small: identify repetitive, well-defined tasks in your organization. Build agentic workflows for those first. As you gain confidence, expand to more…
Multi-agent systems address these by distributing work across specialized agents. Multi-agent systems are the future of enterprise AI. Choose patterns that…
The age of autonomous AI is here. Let's build responsibly.
Fabric AI Skills, Copilot for Microsoft 365, and Copilot Studio all contribute to this trend.
OneLake AI Workloads bring AI capabilities directly to your data, eliminating data movement and enabling efficient large-scale AI processing.
The Fabric + Copilot Studio integration makes enterprise data conversationally accessible while maintaining security and governance.
This unification simplifies the developer experience and provides a clear path from experimentation to production.
Online inference requires careful attention to latency, reliability, and scalability. Choose patterns based on your specific requirements and constraints.
Streaming ML enables intelligent real-time systems that continuously learn and adapt. Start with robust feature engineering and monitoring before enabling…
Real-time AI enables immediate, intelligent responses to streaming data. Start with well-defined use cases and gradually increase complexity.
Eventstream enhancements make real-time analytics more powerful and accessible. Start with simple streaming scenarios and progressively add complexity.
Think of it as the "Visual Studio for AI" - one place to build everything from simple chatbots to complex multi-agent systems.
Analytics agents reduce manual toil while improving data reliability. Start with monitoring agents for quick wins, then expand to exploration and quality…
AI Skills democratize data access while maintaining governance. Start with well-defined domains and expand based on user feedback.
The Fabric platform continues to mature into a comprehensive, AI-native analytics solution.
Start with declarative agents for rapid prototyping and move to custom engines when you hit limitations.
Copilot for Microsoft 365 represents a fundamental shift in how knowledge workers interact with productivity tools. The key is thoughtful adoption that…
Actions in Copilot Studio are implemented using Power Automate flows or Azure Functions. Multi-step workflows are best implemented using Power Automate flows.
Copilot Studio bridges the gap between low-code simplicity and enterprise requirements. Start with templates and progressively add custom actions as needs…
Microsoft Copilot is becoming the unified AI interface across the Microsoft ecosystem. Understanding these capabilities helps you leverage AI assistants…
Enterprise agents require these patterns to ensure security, compliance, and reliable operation at scale. Implement them from the start rather than…
Templates accelerate agent development by providing battle-tested patterns. Start with a template, customize for your needs, and iterate based on real usage.
Multi-agent systems unlock sophisticated automation capabilities. Start with simple patterns and evolve complexity as needed.
Azure AI Agent Service provides the foundation for building sophisticated AI systems that can reason, plan, and act autonomously. Start with simple agents…
Best for: Variable workloads, quick start, minimal ops overhead Best for: Predictable workloads, SLA requirements, cost optimization
Serverless fine-tuning removes the infrastructure barrier to custom model development. Start experimenting with your domain-specific use cases today.
The Model Catalog gives you flexibility to choose the right model for each use case while maintaining a consistent API. Experiment with different models to…
The key to successful AI applications is choosing the right architecture for your requirements and building with production considerations from the start.
Azure AI Foundry provides the foundation for building enterprise-grade AI applications with proper tooling, evaluation, and monitoring. The platform…
The best preparation is building flexible systems that can quickly adopt new models while maintaining production stability.
The reasoning tokens are where the model works through the problem step-by-step, similar to how humans solve complex problems.
trainingjsonl = preparetrainingdata(trainingexamples) with open("trainingdata.jsonl", "w") as f: f.write(trainingjsonl)
Batch processing can reduce costs by up to 50% for workloads that don't need immediate responses.
The DP-600 certification is your gateway to becoming a recognized Microsoft Fabric expert. Good luck with your exam preparation!
The DP-600 certification demonstrates your expertise in Microsoft Fabric. Focus on hands-on practice and understanding the "why" behind each feature, not…
Stay informed about the Fabric roadmap to plan your data platform strategy effectively. The pace of innovation is rapid, so regular review of announcements…
Microsoft Fabric represents the natural evolution toward unified, AI-powered data platforms. Understanding this evolution helps you make informed decisions…
A unified data platform on Fabric eliminates the complexity of managing multiple systems while providing the flexibility to handle any data workload.
PostgreSQL in Fabric will bring familiar PostgreSQL capabilities to the unified data platform. Start preparing now by designing schemas that work well with…
SQL Database in Fabric brings enterprise-grade transactional capabilities to the unified data platform, with seamless integration to analytical workloads…
Fabric Databases bring operational database workloads into the unified Fabric platform, with automatic integration to OneLake.
Microsoft announced the preview of relational databases directly within Fabric, bringing transactional workloads into the unified platform.
The AI landscape is evolving rapidly. OpenAI has hinted at improved reasoning capabilities in future models. Build your infrastructure to be flexible and…
Pre-filtering: Filter before vector search Post-filtering: Filter after vector search Azure AI Search uses pre-filtering with automatic optimization.
HNSW builds a multi layer graph where: Higher layers : Sparse, for fast long range navigation Lower layers : Dense, for precise local search Layer 0 :…
PQ divides each vector into subvectors and quantizes each independently: For even faster search, combine with inverted file index: Dataset Size n subvectors…
Scalar quantization maps floating point values to integers by dividing the value range into buckets: Understanding quantization error helps set…
With quantization, this can be reduced to 1-2 GB.
Vector search finds similar items by comparing their mathematical representations (embeddings). The key components: Embeddings : Dense vectors representing…
Expanded Vector Dimensions Semantic Ranking Improvements Customer-Managed Keys for Vectors
1. Categorize memories : Different types need different handling 2. Extract automatically : Don't rely on explicit save commands 3. Maintain regularly :…
Semantic routing compares the meaning of a user's query against a set of example utterances. When the query is semantically similar to examples for a…
Latency has multiple components: 1. Network latency : Request travel time 2. Queue time : Waiting for processing 3. Time to first token (TTFT) : Initial…
Quality isn't one dimensional. Consider: Accuracy : Factual correctness Completeness : Covering all aspects Coherence : Logical flow and consistency…
The difference is dramatic. A 1000-token task costs $0.02 with GPT-4o but $0.0002 with GPT-4o-mini.
The simplest approach: fixed rules based on task type. Pros: Simple, predictable, easy to debug Cons: Doesn't adapt, requires manual tuning
Different tasks need different capability levels. Using GPT-4o for simple classifications wastes money.
For organizations already invested in Azure, this eliminates the complexity of managing another vendor relationship.
Performance That Competes Claude 3.5 Sonnet outperforms Claude 3 Opus on most benchmarks while being significantly faster and cheaper. It's positioned as a…
Real-Time Intelligence is Fabric's answer to streaming analytics, combining the power of Azure Data Explorer with the unified Fabric experience.
Build 2024 was the most AI-focused Build ever. The theme: AI is moving from demos to production.
AI agents need memory to maintain context across interactions. Today I'm exploring how to implement effective memory systems.
Code execution is one of the most powerful capabilities for AI agents - and one of the most dangerous. Today I'm exploring how to implement it safely.
Azure AI Agent Service provides: Managed agent runtime No infrastructure to manage Built in tools Code execution, file handling, web search Memory…
Debugging AI applications is different from traditional software. Today I'm exploring tracing and debugging techniques for production AI systems.
Without proper evaluation: Models may hallucinate without detection Quality degrades silently over time Compliance violations go unnoticed User experience…
Prompt Flow has evolved significantly since its introduction. Today I'm exploring the latest improvements for building production-ready AI pipelines.
Azure AI Studio now provides a single pane of glass for: Model catalog browsing Prompt engineering Fine tuning Evaluation Deployment Monitoring
Phi-3 uses high-quality training data (textbooks, filtered web content) rather than raw internet scale.
GPT-4o is 2x faster than GPT-4 Turbo. Today I'm exploring how to leverage this speed for responsive applications.
GPT-4o is already 50% cheaper than GPT-4 Turbo, but there are more ways to optimize costs. Here are practical strategies I use in production.
1. Navigate to your Azure OpenAI resource 2. Go to Model deployments Deploy model 3. Select from the model list 4. Configure deployment settings 5. Deploy…
GPT-4o's vision capabilities are impressive, but what makes them practical for enterprise is the combination of quality, speed, and cost. Today I'm…
Voice AI is transforming how we interact with applications. Today I'm exploring how to build voice-enabled AI applications using Azure's current capabilities.
Microsoft has been rapidly expanding Azure OpenAI capabilities. We can expect: New model deployments : Potentially GPT 4 improvements or new multimodal…
Azure Databricks continues to evolve with powerful AI/BI features. April 2024 brings significant updates that blur the line between data engineering and…
Microsoft Fabric continues to evolve as a unified analytics platform. April 2024 brings significant updates across data engineering, science, and business…
March 2024 was a transformative month for AI. Here's a comprehensive recap of the key developments and what they mean for practitioners.
Deploying custom AI models to production requires careful consideration of scalability, reliability, and cost. This guide covers the complete journey from…
Azure OpenAI fine-tuning is now generally available, bringing enterprise-grade model customization to the Azure platform. This comprehensive guide covers…
Fine-tuning allows you to customize LLMs for specific tasks. This guide compares the major approaches available today.
Meta's Llama 2 70B represents the state of the art in open-source large language models. Available on Azure AI, it offers a compelling alternative to…
Cohere's models are now available on Azure AI, offering specialized capabilities for enterprise search and retrieval-augmented generation (RAG). This guide…
Mistral Large is now available on Azure AI, bringing one of Europe's most capable AI models to the Azure ecosystem. This guide covers deployment, usage, and…
The Azure AI Model Catalog continues to expand with new models and capabilities. This month brings significant updates including new foundation models and…
1. Use appropriate features Only request what you need 2. Handle confidence scores Filter low confidence results 3. Combine with GPT 4V Vision 4.0 for…
1. Start with benchmarks Measure actual performance 2. Plan for peaks 2 3x average for safety 3. Build in headroom 20 30% buffer 4. Monitor continuously…
Azure ML's new feature store and Prompt Flow integration changed how I structure ML pipelines in early 2024. Below are the updates that matter operationally…
I've been integrating Azure AI Search into RAG systems; the January 2024 updates simplify common workflows. Below are the changes I judged most impactful…
Capacity choices in Fabric determine both performance and cost. From real projects, I'll share the monitoring signals and optimization steps that actually…
I've been tracking Fabric updates closely since GA in November 2023. In January 2024 several features landed that change operational workflows; below are…
Microsoft's 2023 felt like an all-in moment: platform bets, product integration, and commercial commitments that reshaped the enterprise AI landscape. From…
The Azure OpenAI Assistants API — launched in preview on Azure shortly after the OpenAI DevDay announcement in November 2023 — is the stateful AI agent…
Llama 2 on Azure arrived through the Azure Model Catalog as a managed deployment option, making Meta's open-source family — 7B, 13B, 70B, and their Chat…
Mistral 7B — released by Mistral AI on September 27, 2023 with a permissive Apache 2.0 licence — is the open-source model that forced a recalibration of…
The Azure Model Catalog (available in Azure AI Studio and Azure ML) is where Microsoft is building the answer to the model selection question — "which…
Microsoft Fabric reached General Availability at Ignite 2023 — November 15-17 in Seattle — and it's the outcome I genuinely thought might take another six…
Azure AI Studio arrived at Ignite 2023 as a significantly expanded platform — and "expanded" is the right word, because it builds on Azure ML Studio's…
Fabric licensing is genuinely different from the Azure pay-as-you-go mental model that most Azure practitioners are familiar with, and understanding the…
Microsoft Fabric reaching General Availability at Ignite 2023 (November 15-17) is the milestone that transforms the six-month public preview from an…
RAG with Azure Cognitive Search and Azure OpenAI is the production architecture I recommend most often for enterprise knowledge retrieval — not because it's…
Intelligent document processing is where Azure AI Services earn their keep in enterprise settings — processing invoices, contracts, medical records, forms…
After months of working with Azure OpenAI in enterprise environments, the patterns that separate solid deployments from problematic ones have become clear …
Understanding the key differences between Azure Machine Learning and Azure Cognitive Services, and when to use each service.
A comprehensive guide to building serverless AI solutions using Azure Functions and Azure Cognitive Services.
Exploring the latest Microsoft Fabric updates and how they transform enterprise data analytics.
Learn how to use Azure AI Language's summarization capabilities for extractive and abstractive document summarization.
Build custom NER models to extract domain-specific entities from text using Azure AI Language.
Build custom text classification models using Azure AI Language for domain-specific categorization needs.
Explore the latest updates to Azure AI Language including improved entity recognition, sentiment analysis, and text analytics capabilities.
Build intelligent meeting summarization solutions using Azure AI Speech and OpenAI services.
Build real-time transcription applications using Azure AI Speech services with low latency and high accuracy.
Understanding the ethical considerations and responsible practices for voice cloning and custom neural voice technology.
Learn how to create custom neural voices using Azure AI Speech for branded and personalized voice experiences.
Deep dive into the latest speech-to-text improvements in Azure AI including better accuracy, noise handling, and domain-specific recognition.
Explore the latest updates to Azure AI Speech services including improved speech-to-text accuracy, new voices, and real-time capabilities.
Learn how to analyze video content using Azure AI services for object detection, activity recognition, and content understanding.
Explore the latest updates to Azure Custom Vision including improved training capabilities, edge deployment options, and AutoML features.
Explore Azure's growing catalog of foundation models and how to leverage them for various AI applications.
Exploring the latest Azure AI Services capabilities for vision, speech, and language processing in enterprise applications.
Master prompt engineering techniques for generating high-quality images with DALL-E 2 and other AI image models.
Learn how to generate images using DALL-E 2 on Azure OpenAI Service for creative and business applications.
Learn how to extract structured data from documents using Azure Form Recognizer for intelligent document processing.
Explore Azure Cognitive Services Computer Vision capabilities for image analysis, OCR, and object detection in your applications.
Learn how to use LangChain with Azure OpenAI Service to build robust, production-ready AI applications.
Learn effective strategies for working within GPT-4's context window limits and processing large documents with Azure OpenAI Service.
Learn how to use Microsoft's Semantic Kernel SDK to build intelligent applications that leverage GPT-4 and Azure OpenAI Service.
AI infrastructure costs compound quickly. A few patterns I've been applying with clients to bring Azure OpenAI and Azure ML spend under control: at the API…
Tomorrow we'll explore cost optimization strategies for AI workloads. Azure Spot VMs Checkpointing Best Practices Azure ML Cost Management
Hugging Face and Microsoft's partnership has made the model hub directly accessible from Azure ML, and in August 2023 this is meaningfully useful for Llama…
How you structure workspaces in Fabric matters more than most getting-started guides acknowledge. A workspace is the unit of security, capacity binding, and…
The Fabric trial is one of the more generous evaluation paths Microsoft has offered for a data platform — a 60-day, F64-equivalent capacity for free is a…
I've been onboarding several clients onto Fabric over the past few weeks and the first hour is always the same: getting oriented to an interface that's…
Six weeks into the Fabric public preview and I'm past the demo phase — I'm running actual data engineering workloads and discovering what holds up and what…
Of everything announced at Build 2023, OneLake shortcuts might be the feature I find most architecturally elegant. A shortcut is not a copy, not a linked…
A week after Build 2023, I'm still unpacking what Microsoft Fabric actually means for how we design data platforms. The Lakehouse is the centrepiece — but…
Build 2023 had two dominant themes: 1. AI Everywhere : Generative AI integrated into every product and platform 2. Platform Unification : Simplifying…
Container Apps provides a powerful, serverless container platform. Tomorrow, I will cover Jobs in Container Apps in more detail.
Deployment Environments enables platform engineering teams to provide governed self-service infrastructure to developers. Tomorrow, I will cover Azure…
Dev Box simplifies developer onboarding and ensures consistent development environments. Tomorrow, I will cover Azure Deployment Environments.
At Microsoft Build 2023 on May 23, Satya Nadella announced Microsoft Fabric with the phrase "perhaps the biggest launch of a data product from Microsoft…
All in ONE cohesive experience with ONE security model and ONE business model. At the heart of Fabric is OneLake - think of it as "OneDrive for data." It's…
The fastest approach uses OneLake shortcuts to reference existing data without copying. Run Fabric alongside existing platform during transition.
Both Synapse and Fabric are excellent platforms. The choice depends on your specific requirements, existing investments, and organizational priorities.
A Fabric workspace is the collaboration boundary in Microsoft Fabric—the container where Fabric items (Lakehouses, Warehouses, Notebooks, Pipelines, KQL…
Understanding Fabric licensing helps you plan costs and ensure users have appropriate access. Tomorrow, I will cover Fabric Workspace configuration and…
A Fabric Capacity is a pool of compute resources that powers all Fabric workloads in assigned workspaces. Unlike traditional Azure services where you…
The Fabric Data Warehouse provides enterprise SQL analytics without infrastructure management. Tomorrow, I will cover Synapse Real-Time Analytics for…
The Lakehouse is where most of your Fabric data engineering work will happen. Tomorrow, I will cover Data Factory in Fabric for orchestrating data movement…
OneLake is a single, unified, logical data lake for your entire organization. Think of it as "OneDrive for data" - automatically provisioned when you enable…
This fragmentation creates operational overhead that Fabric aims to eliminate.
Microsoft Build 2023 is scheduled for May 23–25, 2023, in Seattle—and based on the product roadmap signals Microsoft had been sending through Ignite 2022…
Enterprise semantic search transforms how organizations find and use knowledge. By understanding meaning rather than just matching keywords, these systems…
Intelligent document processing transforms unstructured documents into structured, actionable data. Combining OCR, layout analysis, and LLM understanding…
Anomaly detection at scale combines statistical rigor with AI understanding. Systems that both detect and explain anomalies enable faster, more confident…
SynapseML democratizes distributed machine learning. Train on massive datasets, tune hyperparameters in parallel, and deploy models at scale - all with…
Cognitive Services in Synapse brings AI capabilities directly into your data engineering workflows. Process millions of records with sentiment, entities…
Azure Synapse with OpenAI transforms data warehousing from technical SQL expertise to conversational data access. Business users can explore enterprise data…
Azure Databricks with Azure OpenAI creates intelligent data platforms where natural language becomes the interface for data engineering tasks.
Microsoft's strategy is clear: every product gets an AI assistant. These aren't separate products but integrated capabilities powered by Azure OpenAI.
The combination of Azure ML's operational capabilities with Azure OpenAI's language understanding creates powerful, production-ready AI systems.
These deployment patterns enable safe, controlled rollouts of LLM application changes. Start with gateway and blue-green patterns, then add canary and A/B…
LLMOps brings discipline to LLM application development. Start with these foundations and iterate as your applications mature.
Fine-tuning is powerful but requires careful data preparation and evaluation. Start with prompt engineering, use RAG when you need sources, and fine-tune…
The revelation that landed with GPT-4's launch wasn't just the capability improvement—it was learning that Microsoft's new Bing Chat had been running on…
Stay updated through Azure OpenAI documentation and Microsoft announcements.
This isn't just incremental improvement - it's a capability threshold crossing. The AI capability curve just jumped. Time to adapt.
On March 13, 2023—the day before OpenAI announced GPT-4—the AI practitioner community was in an unusual state: highly confident that something significant…
Deploy across regions for resilience: Configure Azure API Management for governance: Enterprise Azure OpenAI deployments need: 1. Multi region failover for…
Streaming transforms AI applications from feeling sluggish to feeling responsive. The implementation adds complexity, but the UX improvement is substantial.…
One token is roughly 4 characters in English. A 1000-word document is about 1300 tokens. Start with these strategies and refine based on your usage…
Vectors (embeddings) represent meaning in high-dimensional space. Similar items have similar vectors.
Azure Form Recognizer transforms manual document processing into automated workflows. The combination of pre-built and custom models handles most document…
Latency : Process locally instead of round trips to Azure Offline capability : Work without internet connectivity Data residency : Keep data on premises…
Responsible AI isn't just about compliance - it's about building trust with users and ensuring AI benefits everyone. Azure OpenAI provides a foundation, but…
Combined with Azure OpenAI's enterprise security and compliance, it's a powerful combination.
RAG is the bridge between general-purpose LLMs and your specific enterprise data. Get it right, and you unlock tremendous value.
Provide specific line-by-line feedback.""", requiredvars=["databasetype", "query"] ) prompt = SQLREVIEWTEMPLATE.format( databasetype="Azure SQL Database"…
Embeddings are numerical representations of text that capture semantic meaning. Similar concepts have similar vectors. Azure OpenAI provides the…
1. Set appropriate thresholds : Adjust based on your use case 2. Use blocklists : For domain specific terms 3. Check both input and output : For AI…
1. Start with fairness : Measure before deploying 2. Document everything : Model cards are essential 3. Monitor continuously : Fairness can drift 4. Include…
GPT 4 integration in Azure OpenAI More neural voice options Enhanced document understanding Improved multi modal capabilities What's New in Azure AI Azure…
1. Use layout for structure : When you need document organization 2. Handle multi page tables : Merge tables split across pages 3. Validate table structure…
1. Choose the right model : Use specific models for better accuracy 2. Handle confidence scores : Filter low confidence extractions 3. Validate extracted…
Combine multiple models for different document types: 1. Use 5+ training samples : More samples improve accuracy 2. Include edge cases : Train on variations…
Use prebuilt models for common document types: Extract document structure: Extract and process tables: Train models for your specific documents: Use…
1. Organize into collections : Separate by topic/domain 2. Use meaningful IDs : Enable updates and deletions 3. Set appropriate relevance thresholds : Too…
Creates multi step plans: Selects the single best action: Iteratively reasons through complex problems: 1. Register clear skill descriptions : Planners rely…
1. Single responsibility : Each plugin should do one thing well 2. Clear descriptions : Help planners understand what functions do 3. Validate inputs :…
Define AI functions using natural language prompts: Create reusable prompts: Add Python functions as skills: Combine functions into pipelines: Store and…
Create reusable, parameterized prompts: Combine components into workflows: Load content from various sources: Split documents for embedding: Generate…
Cross encoders directly score query document pairs: Use an LLM to judge relevance: Using Cohere's specialized rerank API: 1. Retrieve more, re rank fewer :…
A robust way to combine rankings: Adjust weights based on query characteristics: 1. Start balanced : 50/50 is often a good default 2. Tune on your data :…
OpenAI Embeddings Guide MTEB Benchmark Embedding Best Practices
Simple but effective for uniform content: Respect sentence boundaries: Natural document structure: Use embeddings to find natural break points: Hierarchical…
The basic pattern for straightforward use cases: Transform queries for better retrieval: Generate a hypothetical answer first, then use it for retrieval:…
RAG solves both by retrieving relevant context before generating responses.
Native Azure Integration : Works seamlessly with Azure services Hybrid Search : Combine vectors with full text search Enterprise Ready : Security,…
Traditional databases are optimized for exact matches and range queries. Vector search requires finding approximate nearest neighbors in high-dimensional…
Combine semantic search with keyword matching: Expand queries for better recall: 1. Pre compute embeddings : Don't embed at query time for documents 2. Use…
Embeddings are dense vector representations of text where: Similar meanings are close together in vector space Different meanings are far apart…
Without streaming, users wait for the entire response: 500 tokens at 50 tokens/second = 10 seconds of waiting Users see nothing, then everything at once…
Azure OpenAI REST API follows this pattern: Two authentication methods are available: Always specify the API version: 1. Always handle errors : Check status…
Azure OpenAI Service gives you API access to OpenAI's models - GPT-3.5, Codex, and DALL-E - but running on Azure infrastructure with enterprise-grade security.
1. Use Azure AD authentication for production 2. Implement retry policies with Polly 3. Use dependency injection for service management 4. Stream responses…
The Python SDK for Azure OpenAI (openai package with Azure-specific configuration) is the de facto starting point for Azure OpenAI development—the largest…
The original API for text generation: Message based API with roles: Create an abstraction that works with both APIs:
Temperature controls randomness in token selection: Temperature = 0 : Nearly deterministic, always picks the most likely token Temperature = 1 : Standard…
System prompts are special messages that set the context for the entire conversation: A well structured system prompt includes: Build a library of reusable…
Chain-of-thought prompting encourages models to break down problems into intermediate reasoning steps before giving a final answer.
Few shot learning provides examples that demonstrate the desired input output pattern: Zero shot : No examples, just instructions One shot : One example Few…
Prompt engineering in January 2023 was the skill that separated AI features that worked in demos from AI features that worked in production. The gap: a demo…
Tokens are the basic units of text that LLMs process. In English: 1 token ≈ 4 characters 1 token ≈ 0.75 words 100 tokens ≈ 75 words Build a comprehensive…
Azure OpenAI uses Tokens Per Minute (TPM) as the primary quota metric: Model Default TPM Max TPM (with increase) GPT 3.5 Turbo 120K 300K+ Text Davinci 003…
Each category has severity levels: safe, low, medium, high.
Microsoft's Responsible AI framework guides Azure OpenAI Service: 1. Fairness : AI systems should treat all people fairly 2. Reliability & Safety : AI…
The right choice depends on your specific requirements. For most enterprise scenarios, Azure OpenAI's security and compliance features make it the clear…
The question I kept hearing from enterprise clients in January 2023 was some variation of: "We've seen what ChatGPT can do—how do we get that capability…
Until then, these patterns will help you build production-ready chat experiences.
Redis is a popular in-memory data store that can be used as a cache. Azure Functions is a serverless compute service that can be used to run code on-demand…
This will install the Azurite package globally, allowing you to use it from the command line.
GPT 3.5 is not a single model but a family of models with different capabilities: Model Best For Max Tokens Cost text davinci 003 Complex tasks, longer…
2023 is going to be the year AI goes mainstream in enterprise applications. Azure OpenAI Service removes the last major barrier - security and compliance…
Closing out 2022 from Australia on New Year's Eve, the technical predictions for 2023 feel more consequential than they ever have—because November 30…
Containerization and Kubernetes: Lessons Learned in 2022
Observability in 2022 moved beyond simple monitoring to true understanding of system behavior. OpenTelemetry became the standard for instrumentation. SLOs…
2022 established these patterns as the foundation of modern data engineering. The lakehouse became the default architecture for new data platforms. Data…
IaC maturity is a journey, not a destination. Start where you are, establish good practices early, and continuously improve. The goal is infrastructure that…
2022 brought significant security advancements in Azure, from Defender for Cloud improvements to enhanced Zero Trust capabilities. As we move into 2023,…
These scripts are my gift to you for a more efficient 2023. Start with the cleanup script to reduce waste, then implement tag compliance and scheduling.…
1. Collect metrics CPU, memory, network, IOPS 2. Analyze patterns Peak, average, trends 3. Identify candidates Under utilized resources 4. Recommend changes…
I'm very surprised that there isn't a straight forward library out there that is free to use and convenient in terms of file conversion. I think it's…
Auto scaling is powerful but requires careful tuning. Start with conservative settings, monitor behavior, and adjust based on real data. Combine metric…
Spot VMs use Azure's excess capacity at significant discounts. Key characteristics: Up to 90% discount Can be evicted anytime 30 second eviction notice Best…
Azure offers reservations for: Virtual Machines Azure SQL Database Cosmos DB Storage Azure Synapse Analytics App Service Azure Databricks Azure Cache for…
Cost optimization is an ongoing practice, not a one time project. The combination of right sizing, reservations, storage optimization, and operational…
1. Carbon : Reduce the carbon emissions of software 2. Electricity : Reduce the electricity consumption 3. Carbon Intensity : Run workloads when/where…
1. Teams need to collaborate : Finance, engineering, and business work together 2. Everyone takes ownership : Cost is everyone's responsibility 3. A…
The convergence of data lakes and data warehouses became mainstream: Streaming data processing became more accessible: Organizations adopted data mesh…
The biggest story of the year bringing GPT 3, Codex, and DALL E to enterprise: General availability expanded New models including GPT 3.5 Content filtering…
Container Apps sits between App Service and AKS: Simpler than AKS : No cluster management, no node pools More flexible than App Service : Any container, any…
The partition key decision is crucial and difficult to change later: New in 2022 partition up to 3 levels: Optimize indexing for your query patterns: Write…
Don't wait for auto scaling to catch up during traffic spikes: Reduce database load with intelligent caching: Use read replicas and query optimization:…
Enterprise features like private endpoints, managed identity, and content filtering make these models production-ready.
Self service analytics : Users can create their own reports Modern visualizations : Interactive, mobile friendly reports Cloud first : Scales without…
Lift and shift existing SSIS packages with minimal changes: Convert SSIS logic to native Azure Data Factory: SSIS Component Data Flow Equivalent OLE DB…
Synapse Pathway translates SQL code from various sources to Azure Synapse Analytics SQL syntax: Teradata to Synapse Netezza to Synapse Snowflake to Synapse…
Azure Synapse Analytics continues to mature as a comprehensive analytics platform. The improvements in serverless SQL, Spark integration, and data lake…
Bot Framework Composer now supports more advanced dialog patterns: Azure Bot Service provides a comprehensive platform for building conversational AI. With…
Cognitive Services now includes Azure OpenAI Service, bringing GPT models into the Cognitive Services family: The new Image Analysis API includes enhanced…
Foundation models are pre trained on massive datasets and can be: Used directly for various tasks Fine tuned for specific domains Used as feature extractors…
When building applications with GPT 3 and other LLMs, you need to think about: Prompt design and management Chaining multiple LLM calls Testing and…
Feature stores are crucial for ML operations. Azure ML now includes a managed feature store: The Responsible AI dashboard now includes more capabilities:…
DALL E 2 creates realistic images from natural language descriptions. It can: Generate original images from text prompts Edit existing images based on…
return self.generate(prompt, maxtokens=500, temperature=0.3) return self.generate(prompt, maxtokens=500, temperature=0.2)
Provide the optimized query with explanations.""" ERRORDIAGNOSIS = """Diagnose this error: Error: {error} Context: {context}
Azure OpenAI Service is moving toward broader preview access, with more customers being able to apply and get approved. The application process is becoming…
JSON transcoding bridges this gap.
Output caching stores the complete HTTP response and serves it directly for subsequent identical requests, bypassing the entire request pipeline. Unlike…
Rate limiting protects your application by: Preventing denial of service attacks Ensuring fair usage among clients Protecting downstream services Managing…
The biggest addition filters provide middleware like functionality for individual endpoints: You can create reusable filter classes: Organize related…
The isolated worker process model separates your function code from the Azure Functions host: Version independence : Use any .NET version, including .NET 7…
.NET 7 continues the tradition of making .NET faster and more cloud-native with each release. Here's a preview of the key improvements we'll see at GA.
Azure DevOps provides a comprehensive platform for enterprise DevOps with robust security and governance features.
Self-hosted agents provide flexibility for specialized build requirements and secure access to internal resources.
Agent pools provide flexible compute resources for diverse build requirements.
Secure Files protect sensitive credentials while enabling automated pipelines.
Variable groups simplify configuration management across multiple pipelines.
Service connections provide secure, manageable access to external resources.
Checks provide automated validation gates for enterprise deployments.
Approvals and checks ensure deployments meet organizational requirements.
Gates automate quality assurance in your deployment pipelines.
Environments provide governance and visibility for deployments across your infrastructure.
Azure Pipelines templates and stages enable maintainable, scalable CI/CD implementations.
Azure DevOps provides enterprise-grade DevOps capabilities with deep Azure integration.
Azure Orbital brings space data directly into the Azure ecosystem for processing and analysis.
Azure Private MEC delivers cloud capabilities at the edge with enterprise-grade security and management.
Edge computing with Azure provides the foundation for intelligent, responsive, and resilient IoT solutions.
Azure Stack HCI is a hyperconverged infrastructure solution for running virtualized workloads on-premises with Azure integration.
Hybrid cloud strategies provide flexibility while meeting compliance and performance requirements.
Official site : ignite.microsoft.com Tech Community : techcommunity.microsoft.com Learn : learn.microsoft.com YouTube : Microsoft Mechanics channel Stay…
Azure SQL bindings simplify serverless database applications significantly.
SQLCMD mode transforms Azure Data Studio into a powerful deployment and scripting platform.
DACPACs provide reliable, repeatable database deployments for enterprise applications.
Schema Compare is essential for maintaining database consistency across environments.
Database Projects bring modern DevOps practices to database development.
The query editor in Azure Data Studio provides everything needed for efficient SQL development.
Extensions transform Azure Data Studio into a comprehensive database development platform.
Notebooks consist of cells that can contain either code or markdown text. Code cells can execute SQL, Python, PowerShell, or other languages depending on…
Azure Data Studio offers a lightweight, customizable interface that combines the best features of modern code editors with database-specific functionality.
SQL Agent in Managed Instance provides enterprise-grade job scheduling without managing infrastructure.
Service Broker provides queuing and messaging functionality within SQL Server, allowing for decoupled, asynchronous communication between services.
Applying CQS (Command Query Separation) in Azure Functions is a pattern I've found genuinely useful for keeping serverless code maintainable as the number…
Distributed transactions coordinate changes across multiple databases using the two-phase commit protocol. Managed Instance supports both cross-database and…
The Link creates a near real-time data replication from SQL Server (2016 or later) to Azure SQL Managed Instance using distributed availability groups…
Instance pools allow you to host multiple Managed Instances with shared resources, optimizing costs for smaller workloads.
RLS uses security policies with predicate functions to filter rows transparently. Users see only the rows they're authorized to access without any…
DDM is a policy-based security feature that hides sensitive data in query results. The data in the database is not modified, making it ideal for scenarios…
Always Encrypted uses column encryption keys (CEKs) protected by column master keys (CMKs) to encrypt sensitive columns. The database engine never has…
In one of the small projects I'm working on Rust, I actually need to send huge amount of messages on to the cloud and Azure Event Hubs is my preferred…
Ledger tables maintain a complete history of all changes with cryptographic hashes, making it possible to verify that data hasn't been tampered with.
Not to confuse yourself with Xamarin.Mac and MAUI for Mac Catalyst, they are different from this. Xamarin.Mac is an abstraction on a lot of Cocoa based…
Azure SQL Database's continuous improvements make it an excellent choice for cloud-native applications requiring robust, scalable database capabilities.
Exploring the local neighborhood of a vertex is one of the most common operations in graph databases.
Mastering Gremlin opens up powerful graph analytics capabilities in Azure Cosmos DB.
Graph databases unlock powerful relationship-based queries that would be complex or impossible with traditional relational approaches.
Graph databases excel at representing relationships between entities. Unlike relational databases where joins can become expensive, graph databases traverse…
Azure Cosmos DB for Apache Cassandra bridges the gap between familiar Cassandra development and cloud-native scalability.
Azure Cosmos DB's MongoDB API provides a powerful option for teams wanting MongoDB compatibility with Azure's enterprise-grade platform.
PITR works with the continuous backup feature to provide granular recovery options. You can restore entire accounts, specific databases, or individual…
Unlike periodic backup mode, continuous backup captures every change as it happens, allowing granular point-in-time recovery down to the second.
When your Cosmos DB container isn't fully utilizing its provisioned throughput, the unused capacity accumulates as burst credits. These credits can be used…
Traditional single partition keys can lead to hot partitions in scenarios where data is naturally hierarchical. Hierarchical partition keys solve this by…
The latest updates bring significant performance enhancements to Cosmos DB, including optimized query execution and improved throughput management.
Custom skills unlock unlimited possibilities for AI enrichment tailored to your specific business needs.
Skillsets enable powerful AI-driven content enrichment during the indexing process.
Debug sessions are essential for developing and troubleshooting complex AI enrichment pipelines efficiently.
Enrichment cache significantly reduces costs and improves performance for AI-enriched search solutions.
Projections enable flexible storage and querying of AI-enriched content for various downstream use cases.
Knowledge Store enables rich analytics and knowledge graph construction from AI-enriched content.
Hybrid search patterns enable building search systems that handle diverse query types effectively.
Vector search enables powerful similarity-based retrieval that complements traditional keyword search.
Semantic search dramatically improves search relevance by understanding user intent rather than just matching keywords.
Azure Cognitive Search provides enterprise-grade search capabilities with AI enrichment options.
AutoML accelerates model development while maintaining production-quality results.
Parallel jobs enable processing datasets of any size by distributing work across your compute cluster.
Sweep jobs automate the tedious process of hyperparameter tuning, helping you find optimal configurations faster.
Well-designed components enable team collaboration and accelerate ML development through reuse.
Azure ML Pipelines v2 provides a modern, Pythonic way to build production-ready ML workflows.
Prediction drift monitoring provides early warning of model issues without waiting for ground truth labels.
Feature-level drift monitoring enables targeted investigation and remediation of model issues.
Detecting concept drift enables timely model retraining to maintain prediction accuracy over time.
Early drift detection enables proactive model maintenance and prevents silent failures in production.
Comprehensive monitoring ensures your ML models maintain their performance and reliability in production.
A/B testing provides statistical evidence for model selection based on actual business outcomes.
Canary deployment provides a controlled, observable approach to rolling out new model versions with minimal risk.
Blue-green deployment provides a safe, zero-downtime approach to updating ML models in production.
Managed online endpoints simplify model deployment while providing enterprise-grade reliability and scalability.
Causal inference enables data-driven decision making by understanding the true impact of interventions.
Counterfactual analysis makes ML models more interpretable and provides actionable guidance for users.
Error analysis reveals the weaknesses in your model and guides targeted improvements.
Fairness assessment ensures your ML models treat all groups equitably and comply with ethical standards.
Model explanations build trust in AI systems and help identify areas for improvement.
The RAI Dashboard enables you to build and deploy ML models that are fair, interpretable, and reliable.
Azure Machine Learning SDK v2 provides a cleaner, more intuitive API for the complete ML lifecycle.
This architecture provides a scalable, secure foundation for enterprise IoT solutions on Azure.
Implementing these security best practices creates a defense-in-depth approach for your IoT solutions.
Device twins provide the foundation for managing device state, configuration, and organization in IoT solutions.
Effective use of IoT Hub queries enables proactive device fleet management at scale.
Message enrichments simplify downstream processing by providing device context at the point of ingestion.
Custom endpoints enable you to build flexible, scalable IoT data processing architectures tailored to your specific requirements.
The built-in endpoint provides a reliable, scalable way to consume IoT device telemetry for real-time processing.
Message routing is the foundation for building sophisticated, event-driven IoT architectures on Azure.
DPS is essential for deploying and managing IoT devices at scale across multiple regions and IoT Hubs.
These IoT Hub updates enable more sophisticated device management and data processing scenarios.
Handle intermittent connectivity by storing data locally and forwarding when connected. Reduce data volume by aggregating at the edge before cloud sync.
These patterns form the foundation for building scalable, performant IoT database solutions.
Azure SQL Edge enables intelligent edge computing with the familiar SQL Server programming model.
The Link feature for Azure SQL Managed Instance enables replicating databases from SQL Server 2019 and SQL Server 2022 to Azure SQL Managed Instance in…
SQL Server 2022 entered public preview in late 2021 and continued preview through 2022 with a feature set deliberately designed to bridge the gap between…
Azure Arc-enabled SQL Managed Instance delivers enterprise SQL Server capabilities with cloud-native operations on any infrastructure.
Azure Arc-enabled PostgreSQL Hyperscale brings the best of Azure's managed database services to wherever your data needs to live.
Flexible Server gives you the power of managed PostgreSQL with the flexibility to tune it for your specific needs.
Read replicas in Azure Database for MySQL Flexible Server enable scaling out read-heavy workloads by directing reporting queries, analytical queries, and…
Implementing these HA patterns ensures your MySQL applications remain resilient during infrastructure failures.
MySQL 8.0 features significantly enhance your application capabilities on Azure Database for MySQL Flexible Server.
Flexible Server provides the enterprise features you need while maintaining the simplicity of a managed service.
This architecture handles billions of time-series data points while maintaining query performance for both real-time and historical analysis.
Combining Citus's distributed query power with rollup tables and caching enables dashboards that handle millions of events per second.
When two distributed tables share the same distribution column and colocation group, their corresponding shards are placed on the same worker node.
Understanding these query patterns helps you leverage Citus effectively for high-performance distributed PostgreSQL applications.
Reference tables eliminate network round-trips for joins, significantly improving query performance in distributed setups.
The distribution column (also called the shard key) determines how data is partitioned across nodes. Citus uses consistent hashing on this column to assign…
Understanding this architecture helps you make informed decisions about data modeling and query optimization in your distributed PostgreSQL deployments.
Azure Cosmos DB for PostgreSQL is a managed database service that extends PostgreSQL with distributed capabilities. It uses the Citus extension to transform…
Pattern matching in streaming data enables: Sequence detection Anomaly identification Business rule enforcement Fraud detection Predictive maintenance Azure…
Unity Catalog provides a unified governance layer across all your Databricks workspaces. Metastore: The top-level container for all metadata. You create one…
Temporal joins and windows enable sophisticated event correlation, essential for IoT analytics, fraud detection, and user behavior analysis.
Static : Loaded once at job start Periodic : Refreshes at intervals Dynamic : SQL queries for latest data Reference data joins combine streaming data with…
The new visual editor allows building streaming pipelines without SQL: 1. Drag and drop input/output connections 2. Visual transformations and aggregations…
Azure's real time analytics stack enables instant insights from millions of events per second, powering live dashboards and alerts.
Azure Web PubSub offers native WebSocket support with powerful server side messaging APIs, ideal for IoT, gaming, and collaboration apps.
Azure SignalR Service scales real time applications effortlessly, handling millions of connections with automatic load balancing.
Feature SSE WebSocket Direction Server to client Bidirectional Protocol HTTP WebSocket Reconnection Automatic Manual Binary data No Yes SSE offers a simple,…
GraphQL subscriptions provide elegant real time communication, ideal for dashboards, notifications, and live updates.
APIM GraphQL support enables modern API patterns with enterprise governance, rate limiting, and authentication.
gRPC offers binary serialization, HTTP/2 multiplexing, and streaming for high performance microservices on Azure.
Open Service Mesh: Azure's SMI-Compatible Mesh
Traditional Azure CNI assigns VNet IPs to pods, limiting scale. Overlay mode uses a separate address space for pods.
Dockershim removal Seccomp by default Pod security admission Ephemeral containers GA Storage capacity tracking With Dockershim removed, containerd is now…
Cloud based developer workstations with instant provisioning: Pre configured development environments Self service portal for developers Cost management…
Common optimization problems suitable for quantum: Traveling Salesman Problem (TSP) Vehicle Routing Portfolio Optimization Job Shop Scheduling Max Cut…
Azure Quantum provides several simulator options: Simulates all 2^n amplitudes: Run with the full state simulator: Efficient for states with few non zero…
Setting up a Q project: Q programming provides: High level quantum abstractions Built in resource management Classical quantum hybrid programs Comprehensive…
Key concepts: Qubits : Quantum bits that can be in superposition Superposition : Existing in multiple states simultaneously Entanglement : Correlated…
Microsoft's partnership with space industry providers enables access to various satellite data: Use Azure Functions for image processing: Analyze vegetation…
Azure Orbital offers: Global ground station network Satellite communication management Data processing pipelines Integration with Azure services Configure…
Configure diagnostic settings to export to Data Lake: Use serverless SQL to query exported logs: Complex log analysis: Process logs with Azure Functions:…
// Enable streaming ingestion .alter database SensorAnalytics policy streamingingestion enable
Connect operational data to analytics: Enhanced query capabilities: Use Spark for complex analytics: Time series and log analytics: Build data pipelines:…
Custom connectors bridge the gap between your APIs and the Power Platform ecosystem, enabling: Power Apps integration Power Automate workflows Logic Apps…
Office Add ins use web technologies and run across platforms: Create an Excel add in for data analysis: Create a Word add in for document processing: Create…
Install and configure the Microsoft Graph SDK: Query and manage user data: Manage calendars and events: Access OneDrive and SharePoint files: Optimize…
Teams apps can include: Tabs (embedded web content) Bots (conversational interfaces) Message extensions Webhooks and connectors Meeting extensions Create an…
Voice and video calling Chat messaging SMS messaging Email communication Phone number management Create and configure your Communication Services resource:…
Generic math interfaces Required members Improved minimal APIs Native AOT compilation Enhanced performance One of the most anticipated features is generic…
Create a new Function App with Node.js 18: Update your configuration: Azure Functions v4 introduces a new programming model: Use the built in fetch without…
Build 2022 brings several enhancements: Stable APIs using Azure Functions Enterprise edge support Improved authentication options Split environment…
Dapr (Distributed Application Runtime) is a portable, event-driven runtime that makes it easy to build resilient, microservice applications. It provides…
Unlike AKS where you manage the cluster, Container Apps abstracts away the infrastructure. You focus on your containers while Azure handles scaling, load…
GitHub Codespaces provides browser-accessible, cloud-hosted VS Code development environments where the development container (with all tools, extensions…
Azure Deployment Environments addresses these challenges with templated, self-service infrastructure.
Microsoft Dev Box provides self-service, high-performance, cloud-based workstations pre-configured with project-specific tools, source code, and settings.…
Power BI Embedded enables rich analytics in any application: Interactive reports and dashboards Secure multi tenant access Programmatic control and…
Cloud powered RPA with Azure provides: Scalable bot infrastructure Queue based work distribution Centralized monitoring Enterprise grade security Combining…
March 2022 was a productive month for the Azure data and AI ecosystem: the Azure OpenAI Service access expansion brought GPT-3 and Codex to more enterprise…
1. Document all forwarding rules : Maintain a central registry 2. Use redundant DNS servers : Always specify multiple targets 3. Monitor DNS query latency :…
DNS Private Resolver solves this without virtual machines.
Private Endpoint : Consume services privately (you're the client) Private Link Service : Expose services privately (you're the provider) 1. SaaS providers :…
A private endpoint: Creates a network interface with a private IP in your VNet Maps to a specific Azure resource (or sub resource) Enables DNS resolution to…
Data Flows automatically use the managed VNet when connected to private endpoints.
For large datasets, use streaming to incrementally update: Materialized views in Delta Live Tables provide: Precomputed results for fast query performance…
DLT streaming tables use append mode by default. For aggregations: DLT handles checkpointing automatically: Streaming tables in Delta Live Tables provide:…
DLT's applychanges function processes these into clean, current-state tables.
The expression must evaluate to true for valid records.
Traditional ETL: Delta Live Tables: DLT provides built in data quality enforcement: Handle CDC feeds with APPLY CHANGES: Delta Live Tables transforms data…
Key features: Jobs : Multi task workflows with dependencies Triggers : Schedule, file arrival, or API based Compute : Job clusters or serverless Monitoring…
Databricks AutoML automatically: Prepares and preprocesses data Engineers features Selects algorithms Tunes hyperparameters Evaluates and compares models…
The registry organizes models with: Registered Models : Named model artifacts Model Versions : Specific iterations of a model Stages : Lifecycle states…
Key features: Serverless : No infrastructure management Auto scaling : Handles variable traffic automatically Low latency : Sub second response times…
The Feature Store solves these problems.
Think of it as the best of both worlds: visual design for maintainability, Spark for scale.
The ML platform includes: Feature Store : Centralized feature management AutoML : Automated model training and selection MLflow : Experiment tracking and…
Photon is Databricks' native vectorised query execution engine written in C++—a replacement for the JVM-based Apache Spark execution engine for SQL and…
Databricks SQL offers: SQL Endpoints : Serverless compute for SQL queries Query Editor : Web based SQL IDE Dashboards : Built in visualization and reporting…
The protocol is open source, meaning recipients don't need Databricks to access shared data.
A complete governance framework addresses: Access Control : Who can access what data Data Quality : Ensuring data accuracy and completeness Data Lineage :…
Unity Catalog solves these problems with a unified approach.
The service provides: Smart anomaly detection : ML powered detection without manual threshold tuning Root cause analysis : Automatic correlation across…
The service provides: Read Aloud : Text to speech with word highlighting Text Preferences : Font size, spacing, and background colors Grammar Tools :…
All three happen with minimal latency for real-time conversations.
Custom Neural Voice uses deep neural networks to create natural sounding synthetic voices from audio recordings. Unlike traditional text to speech that…
Form Recognizer v3 introduces: Unified API : Single endpoint for all document types Improved accuracy : Better handling of handwriting and poor quality…
The new Computer Vision API brings significant improvements: Analyze people movement in physical spaces: The new unified Language service includes enhanced…
response = openai.Completion.create( engine="code-davinci-002", prompt=prompt, maxtokens=300, temperature=0.3 )
One of the most immediate applications - turning lengthy documents into concise summaries.
This isn't just "OpenAI but on Azure" - it's OpenAI made enterprise-ready. The approval process exists because Microsoft wants to ensure responsible use. Be…
Prometheus remote write to Azure Monitor provides a powerful, unified metrics platform for cloud-native applications.
Custom logs enable comprehensive observability across all your applications and services.
Comprehensive Windows Event collection is essential for security monitoring and compliance.
Comprehensive syslog collection provides essential visibility into Linux system security and operations.
Plan your migration early to ensure a smooth transition before the deprecation deadline.
Azure Monitor Agent provides a unified, secure, and flexible foundation for all monitoring needs.
Data Collection Rules provide the flexibility and control needed for modern monitoring architectures.
Dedicated clusters provide enterprise-grade security and scale for Azure Monitor Logs.
Azure Managed Grafana provides enterprise observability with seamless Azure integration and reduced operational overhead.
Azure Confidential Ledger provides cryptographically verifiable, tamper-proof record keeping for compliance-critical scenarios.
HashiCorp Vault provides enterprise-grade secrets management with dynamic credentials and multi-cloud support.
Category Example Recommended Approach Azure Services Storage, SQL Managed Identity External APIs Third party APIs Key Vault Certificates TLS, Auth Key Vault…
Automated secret rotation ensures credentials remain secure while reducing operational burden.
Certificate-less authentication dramatically simplifies operations while improving security posture.
Keyless Authentication: The Future of Cloud Security
OIDC for GitHub Actions is the modern, secure approach to Azure authentication in CI/CD pipelines.
Instead of client secrets: 1. External identity provider issues a token 2. Token is exchanged for an Azure AD token 3. Application uses Azure AD token to…
Managed identities are the foundation of a zero-secrets architecture in Azure.
Dataverse integration enables a seamless data platform spanning Power Platform and Azure services.
Data Flows Gen2 democratize data transformation, enabling both developers and data analysts to build scalable ETL solutions.
Properly optimized Spark pools can process petabytes of data efficiently and cost-effectively.
These optimizations can dramatically improve dedicated SQL pool performance while reducing resource consumption.
Serverless SQL charges based on data processed. Optimization reduces both cost and query time.
Azure Synapse Analytics in 2022 provides a more unified, performant, and feature-rich analytics platform.
Microsoft has committed to being carbon negative by 2030. Azure regions are increasingly powered by renewable energy, and new tools help customers measure…
Azure Advisor Score provides actionable insights to continuously improve your Azure environment's health and efficiency.
Regular Well-Architected Reviews ensure your workloads remain optimized across all five pillars as they evolve.
The Microsoft Cloud Adoption Framework (CAF) is the comprehensive guidance body that addresses the full journey of cloud adoption—not just the technical…
Azure Landing Zones provide the foundation for a successful enterprise cloud journey, ensuring consistency, security, and governance from day one.
The move away from Blueprints leads to more flexible, testable, and maintainable infrastructure governance.
Azure Policy as Code ensures consistent, auditable governance across your entire Azure estate.
What-if is an essential safety net for infrastructure deployments, preventing costly mistakes before they happen.
The Bicep Registry is essential for scaling infrastructure as code across enterprise organizations.
Bicep in 2022 is production-ready and the best choice for Azure infrastructure as code.
Azure DevOps in 2022 offers a mature, flexible platform for any DevOps workflow.
If you have Windows Server or SQL Server licenses with Software Assurance, you can bring them to Azure instead of paying for new licenses.
RIs provide a billing discount in exchange for a commitment (1 or 3 years). They don't allocate capacity - they provide a discount on capacity you're…
Smart Spot instance architecture lets you achieve massive cost savings while maintaining the reliability your applications need.
Spot VMs use Azure's spare capacity. When Azure needs the capacity for pay-as-you-go customers, Spot VMs receive a 30-second notice before eviction.
AKS costs come from several components: Virtual Machine nodes Storage (managed disks, Azure Files) Networking (load balancers, bandwidth) Container Registry…
AKS now supports managed identities natively, eliminating the need for service principals: The new workload identity feature provides pod level identity:
Dapr on Azure simplifies building resilient, scalable microservices while leveraging Azure's managed services. The combination provides both portability and…
Azure Container Apps is a fully managed serverless container service that enables you to run microservices and containerized applications without managing…
Predicting Azure's direction for 2022 from the vantage point of January 1st: the Container Apps/serverless container narrative continues to mature (Azure…
Electricity grids mix various energy sources. Carbon intensity varies by: Time of day (solar peaks midday) Weather (wind varies) Demand (peak hours often…
Green computing in 2021 became a real consideration, not just marketing. Azure's commitment to carbon negative by 2030 provides the tools; it's up to us to…
FinOps in 2021 grew from a niche practice to an organizational capability. The tools exist; success requires culture change and executive support.
Cost optimization in 2021 became a core cloud competency. The tools are powerful; success requires discipline and cultural change.
1. Verify Explicitly : Always authenticate and authorize based on all available data points 2. Least Privilege Access : Limit user access with just in time…
IaC maturity in 2021 meant treating infrastructure code with the same rigor as application code. The tools support it; the discipline makes it work.
Event-driven architecture in 2021 moved from architectural pattern to practical implementation. The tools support it, the patterns are proven, and the…
Data engineering in 2021 matured from a support function to a strategic capability. The tools improved, patterns solidified, and the role gained the…
Arc-enabled data services brought Azure SQL and PostgreSQL Hyperscale to any infrastructure, a game-changer for hybrid scenarios.
Ignite Fall 2021 was dense with announcements—Azure Container Apps, Chaos Studio, Load Testing, Service Connector, Azure Developer CLI, Arc-enabled data…
Microsoft Teams integration with Azure enables powerful collaborative applications. Whether building bots, tabs, or webhooks, the combination provides a…
Azure Communication Services enables developers to build rich communication experiences that can interoperate with Microsoft Teams. Whether you need voice…
Azure Container Apps provides a sweet spot between serverless functions and full Kubernetes - the power of containers without the operational complexity of…
Azure Developer CLI represents a significant shift toward developer productivity. By providing high-level commands and proven templates, it reduces the…
Azure Service Connector eliminates the boilerplate of connecting services while implementing security best practices. It's a significant productivity boost…
OpenTelemetry provides a future-proof approach to observability. By using vendor-neutral instrumentation with Azure Monitor as the backend, you get the…
Application Insights provides the deep observability needed for modern applications. Combined with Azure Monitor's broader capabilities, it enables…
Azure Load Testing makes performance validation accessible and integrated into modern DevOps workflows. By catching performance regressions early, you can…
Chaos engineering is the practice of experimenting on a system to build confidence in its ability to withstand turbulent conditions. Netflix pioneered this…
Azure Automanage transforms VM management from a series of manual tasks into an automated, best-practice-driven process. It's ideal for organizations that…
Azure Stack HCI provides a modern hyperconverged infrastructure platform with deep Azure integration. It's ideal for organizations that need on-premises…
Arc-enabled data services bring cloud-native database capabilities to any environment. Whether you need SQL Server or PostgreSQL, you can run managed…
Arc enabled Kubernetes provides: Inventory and grouping : Organize clusters in Azure Resource Manager GitOps configurations : Deploy applications using Git…
Azure Arc-enabled servers bring Azure's management capabilities to your entire server estate, regardless of where those servers run. This unified management…
Azure Percept makes edge AI accessible to developers without deep expertise in hardware or ML. Combined with Azure's cloud services, it enables…
This announcement is exciting for the enterprise AI space. The combination of OpenAI's powerful models with Azure's enterprise infrastructure addresses a…
Azure Synapse Link for Dataverse eliminates the traditional barriers between operational and analytical systems. Business users get insights from live data…
Azure Static Web Apps with enterprise features provides a production-ready platform for modern web applications. The combination of global distribution…
In-Process: Functions run in the same process as the Functions host Isolated Worker: Functions run in a separate worker process
Databricks Repos in production use means the code running in your prod workspace is explicitly linked to a specific Git commit—not "whatever notebooks…
Databricks Git integration (Repos) connects the workspace directly to GitHub, GitLab, Azure DevOps, or Bitbucket, replacing the manual pattern of…
Databricks notebook workflows—chaining notebooks together using dbutils.notebook.run()—are the stepping stone between "notebooks as scripts" and proper…
The Databricks REST API exposes every workspace capability that the UI and CLI provide—and then some—making it the integration point for external…
The Databricks CLI is the command-line interface for Databricks workspace operations—running jobs, managing clusters, deploying libraries, uploading…
The job cluster versus all-purpose cluster decision in Databricks is primarily a cost decision: all-purpose clusters stay running between tasks (you pay for…
Databricks cluster policies are the governance layer that prevents the classic enterprise data platform problem: data scientists spinning up 64-node GPU…
Azure Databricks cluster configuration is where cost and performance trade-offs become very concrete: the wrong cluster type for a workload is either money…
Workload management in Synapse Dedicated SQL Pool is the resource governance system that prevents low-priority ad-hoc queries from consuming all compute and…
Result set caching in Synapse Dedicated SQL Pool stores the output of a query in the dedicated pool's storage and returns the cached result for subsequent…
The COPY command replaced PolyBase as the recommended data loading mechanism for Synapse Dedicated SQL Pool because it's simpler to use and performs at the…
PolyBase is the T-SQL feature that makes Synapse Dedicated SQL Pool a data virtualisation engine, not just a data warehouse—it allows you to define external…
Synapse Dedicated SQL Pool (formerly Azure SQL Data Warehouse) is the massively parallel processing (MPP) engine for structured, petabyte-scale analytical…
Synapse Serverless SQL Pool is the query engine that turned Azure Data Lake into an ad-hoc analytics platform—you write T-SQL, point it at files in ADLS…
Microsoft Ignite Fall 2021 brought a wave of announcements across Azure. While hybrid work and Microsoft Teams dominated the headlines, the data and AI…
External tables in ADX expose data in Azure Blob Storage or ADLS Gen2 as queryable tables without ingesting the data into ADX—useful when the data is too…
Materialized views in ADX pre-aggregate data according to a defined query, updating continuously as new data is ingested—so when you query the view, you're…
KQL functions in Azure Data Explorer are the reusability mechanism that prevents the same complex query logic from being copy-pasted across dashboards…
ADX Continuous Export is the managed pipeline that continuously moves data from ADX tables to Azure Blob Storage or ADLS Gen2—useful for archiving cold data…
Azure Data Explorer is the analytics service that addresses a specific limitation of Log Analytics: KQL query performance on large historical datasets…
Cross-workspace queries in KQL let you write a single query that spans multiple Log Analytics workspaces—essential when your monitoring architecture splits…
Log Analytics workspace design is one of those infrastructure decisions that feels low-stakes until your bill arrives or you hit a query that crosses a…
Azure Monitor for Containers (the Container Insights feature) is the native Azure observability solution for AKS that doesn't require running your own…
Grafana is the visualisation layer that makes Prometheus metrics interpretable at a glance—and for AKS, the starting point is the community dashboards that…
Prometheus became the observability standard for Kubernetes because its data model—time-series metrics with labels—maps naturally to the dynamic…
Container Insights is the Azure Monitor feature that closes the observability gap for AKS clusters—without it, you have Kubernetes metrics available in the…
Azure AD Workload Identity is the architecture-level improvement over AAD Pod Identity that removes the NMI DaemonSet and uses Kubernetes native service…
AAD Pod Identity was the original mechanism for giving Kubernetes pods an Azure AD identity so they could access Azure resources—Key Vault secrets, Storage…
Virtual nodes in AKS enable pods to run on Azure Container Instances rather than on VM-based node pool nodes—useful for burst workloads that spike…
AKS Spot node pools are the cost reduction lever that can cut compute spend by up to 90%—at the cost of accepting that Azure may evict spot nodes with a…
Node pools are the AKS mechanism for running heterogeneous workloads on a single cluster—different VM SKUs, OS types, and node configurations within the…
AKS cluster upgrades are the operational task that feels straightforward until you do them in production and discover the nuances—node cordoning order, pod…
Model GPU Use Case Pro (GPU) NVIDIA T4 AI inference, video analytics Pro (FPGA) Intel Arria 10 Hardware acceleration Pro R Rugged Harsh environments Mini R…
CDN caching rules are the configuration that determines how long cached content stays at the edge before the CDN checks with the origin for updates—and…
Azure CDN significantly improves application performance by reducing latency and offloading traffic from origin servers.
Content protection with DRM ensures your premium content remains secure while providing a seamless viewing experience across all platforms.
Proper streaming endpoint configuration ensures reliable, scalable content delivery to viewers worldwide.
Proper encoding workflow design ensures optimal video quality, efficient delivery, and cost-effective processing for your media applications.
Azure Media Services provides enterprise-grade media processing and delivery capabilities for building streaming platforms and video applications.
Live Video Analytics enables real-time video intelligence at the edge, perfect for security, retail analytics, and industrial monitoring applications.
Azure Video Analyzer brings intelligent video analytics to both edge and cloud, enabling sophisticated spatial analysis and event detection for security…
Document Translation enables global content delivery by making document localization efficient and scalable.
Named Entity Recognition is essential for extracting structured information from unstructured text, enabling applications from search enhancement to…
Key phrase extraction is a foundational NLP capability that enables efficient processing of large text collections and powers intelligent content management…
Sentiment analysis transforms unstructured feedback into actionable insights, enabling data-driven decisions about products, services, and customer experience.
Text Analytics enables rich understanding of unstructured text, powering applications from customer feedback analysis to content recommendation systems.
LUIS enables natural language understanding that powers conversational interfaces, making applications more intuitive and user-friendly.
Speaker Recognition enables secure, frictionless authentication and personalized experiences based on voice identity.
Azure Text-to-Speech enables natural voice experiences across applications, from virtual assistants to accessibility features.
Azure Speech-to-Text is the service I've used in two distinct modes: real-time transcription for live meeting captions and voice-activated applications, and…
IoT Edge with Cognitive Services enables intelligent edge scenarios, bringing AI capabilities directly to where data is generated.
Vision : Read (OCR), Spatial Analysis Language : Sentiment Analysis, Key Phrase Extraction, Language Detection, Text Analytics for Health Speech : Speech to…
Batch endpoints enable cost-effective, scalable inference for large datasets without the complexity of managing infrastructure.
Managed online endpoints provide a robust, production-ready platform for serving ML models with enterprise-grade features out of the box.
Model deployment is where your ML work delivers business value. Azure ML's managed endpoints make it straightforward to deploy, scale, and update models in…
The Model Registry is the cornerstone of production ML. It provides the governance and traceability needed to confidently deploy and manage models at scale.
MLOps transforms ML from an experimental practice to a reliable engineering discipline. Azure ML provides the tools to implement these practices at scale.
Feature stores are becoming essential infrastructure for production ML systems. Understanding these concepts will help you build more reliable and…
Data labeling in Azure ML streamlines the process of creating high-quality training data, essential for building accurate machine learning models.
Proper data management with Azure ML Datasets is foundational to building reproducible, auditable machine learning pipelines.
Compute clusters are essential for production ML workloads. Their ability to scale dynamically and support distributed training makes them the backbone of…
Azure ML Compute Instances are the development environment that removes the "I can't reproduce the team's ML environment" problem for data science teams.…
The Stored Procedure Activity bridges the gap between data movement and data transformation, enabling complex database logic to be orchestrated as part of…
The Web activity in ADF is the HTTP client that lets your pipeline call external REST APIs—triggering a Logic App, posting a Teams notification, calling a…
The Until activity in ADF is the polling loop that keeps running until a condition is true—useful for pipelines that trigger an external process and need to…
The ForEach Activity is the key to building scalable, efficient data pipelines that can process thousands of items in parallel while maintaining control and…
Hybrid cloud is the trend right now I see in a lot of companies and enterprises. The term hybrid has grown too in the last couple of years. It used to be…
The Lookup Activity is the foundation for building intelligent, data-driven pipelines that adapt their behavior based on configuration and runtime data.
Dynamic content in ADF is the expression system that makes pipelines adaptive rather than static—generating file paths from date parameters, constructing…
Parameterized pipelines dramatically reduce maintenance overhead and enable rapid deployment of new data integration scenarios without code changes.
The Copy Activity in Azure Data Factory is the activity I configure more than any other—it's the data movement primitive that connects over 90 source and…
Azure Data Lake Storage Gen2 provides the foundation for modern data analytics, combining the best of blob storage scalability with the directory semantics…
Azure Blob NFS bridges the gap between traditional NFS workloads and cloud-native blob storage, enabling seamless migration of Linux applications without…
Azure HPC Cache bridges the gap between cloud compute and existing storage investments, enabling organizations to run HPC workloads in Azure while…
Azure NetApp Files delivers enterprise storage performance in the cloud, making it the ideal choice for mission-critical applications that demand…
Azure Files is the managed file share service I reach for when an application needs shared file access and rewriting it to use blob storage isn't…
Immutable storage in Azure provides the strong data protection guarantees required for regulatory compliance, ensuring your critical data remains…
Lifecycle management policies automate the tedious work of data tiering and retention, ensuring compliance while optimizing storage costs without manual…
Azure Storage tiers provide a powerful mechanism for balancing performance and cost, enabling organizations to store massive amounts of data economically…
Redis persistence is the configuration choice that determines what happens to your cache data when the instance restarts—and for most Azure Cache for Redis…
Redis clustering provides the horizontal scalability needed for demanding workloads while maintaining Redis's sub-millisecond performance characteristics.
Azure Cache for Redis is the caching layer I add to almost every production Azure application that has read-heavy patterns with data that doesn't change on…
TTL is a powerful feature for automatic data lifecycle management, reducing storage costs and ensuring compliance with data retention policies without…
Proper conflict resolution is essential for multi-region write scenarios. Choose the right strategy based on your data semantics and ensure thorough testing…
Multi-region writes enable true active-active architectures, but require careful design to handle the inevitable conflicts that arise from concurrent…
Global distribution in Cosmos DB provides the foundation for building truly global, resilient applications that serve users with consistent low latency…
Consistency levels in Cosmos DB give you fine-grained control over the CAP theorem tradeoffs, enabling you to optimize for your specific application…
Proper partition key design is fundamental to Cosmos DB success. Take time to analyze your access patterns and data distribution before finalizing your…
Azure SQL's Automatic Tuning is the machine learning system that acts on Query Store data to make index and query plan decisions that would otherwise…
Query Store transforms database performance troubleshooting from guesswork into data-driven analysis, making it an indispensable tool for maintaining…
IQP represents a significant step toward self-tuning databases, reducing the manual effort required to optimize query performance while automatically…
Serverless is perfect for dev/test environments, infrequently used applications, and workloads with predictable quiet periods. For consistent…
The key benefit of elastic pools is resource sharing. If you have 100 databases that each spike at different times, you don't need to provision for 100…
Traditional database architectures couple compute and storage tightly together. Hyperscale breaks this coupling by introducing a distributed storage…
Create a scheduled function: Common CRON patterns: Implement patterns for tasks that may exceed timeout: Ensure single instance execution: Configure…
Queue-triggered Azure Functions are the pattern I recommend most often for decoupling work that doesn't need to happen synchronously. The model is simple…
Understanding when to use each: Feature WebJobs Azure Functions Hosting Part of App Service Standalone or App Service Scaling With App Service plan…
The Azure Functions Premium plan is the hosting option I reach for as soon as two of these three conditions are true: the function needs to connect to a…
DSC works with three key components: Configurations : PowerShell scripts defining desired state Resources : Building blocks for configurations (files,…
Set up Update Management for your VMs: Onboard VMs to Update Management: Schedule update deployments: Create maintenance scripts: Query update compliance…
Set up Azure Automation: Create a runbook for VM management: Create a Python runbook for resource cleanup: Create schedules for automated execution: Create…
Service Health tracks three types of events: Service Issues : Active problems affecting Azure services Planned Maintenance : Upcoming maintenance that might…
Resource Health reports four possible states: Available : Resource is healthy and operating normally Unavailable : Resource is not healthy due to Azure or…
Activity Logs include several categories: Administrative : Resource management operations (create, update, delete) Security : Security Center alerts and…
Azure resources produce several types of diagnostic data: Platform Metrics : Numerical performance data Resource Logs : Detailed operational logs (formerly…
There are two ways to publish custom metrics: Application Insights SDK : For application level metrics Azure Monitor REST API : For any metric source Use…
Azure Monitor collects two types of metrics: Platform Metrics : Automatically collected from Azure resources Custom Metrics : Application specific metrics…
Traditional NSG rules use IP addresses, which creates challenges: IP addresses change when VMs are recreated Rules become hard to read with many IP ranges…
NSG rules are evaluated by priority (100 4096, lower = higher priority): Direction : Inbound or Outbound Priority : 100 4096 (lower numbers processed first)…
Azure Firewall rules are the configuration work that determines whether your network security posture is genuinely restrictive or accidentally permissive.…
Feature Consumption Premium Scaling 0 to 200 instances 1 to 100 instances Cold starts Yes No (pre warmed) Max timeout 10 min (default 5) 60 min (default 30)…
Use queues to handle variable load: Scale processing with multiple consumers: Process high priority items first: Coordinate long running business processes:…
Set up a private DNS zone for your organization: Complete Private DNS setup with Terraform: Create DNS zones for Azure Private Link services: Integrate…
NAT Gateway solves several problems: SNAT Port Exhaustion : Each NAT Gateway supports up to 64,000 concurrent connections per public IP Simplified…
Azure offers two SKUs: Basic : Free, limited features, no SLA Standard : Zone redundant, SLA backed, supports availability zones Set up an internet facing…
VNet peering is the network connectivity primitive I configure in almost every Azure architecture. The pitch is simple: two virtual networks, connected via…
ExpressRoute Direct offers: Direct physical connectivity to Microsoft's network 10 Gbps or 100 Gbps port pairs Support for massive data ingestion scenarios…
Azure offers several Data Box options: Data Box Disk : Up to 40 TB per order (8 TB per disk) Data Box : 100 TB storage capacity Data Box Heavy : Up to 1 PB…
Set up the data sharing infrastructure: Configure datasets to be shared: Invite consumers to access shared data: Set up automated data synchronization: On…
Create and configure an Azure Purview account: Register various data sources for lineage tracking: Set up scans to discover and catalog data: Azure Data…
Power BI Goals arrived in 2021 as Microsoft's answer to OKR tracking inside the BI platform, and I think it was better conceived than it might sound. The…
Premium Gen2 introduces several architectural improvements: Autoscale compute resources based on demand No need for capacity management or monitoring v…
Databricks SQL Analytics provides: Native SQL interface to query Delta Lake tables SQL Endpoints with auto scaling compute Built in visualization and…
The MERGE statement combines INSERT, UPDATE, and DELETE operations in a single atomic transaction: Handle complex business logic with conditional updates:…
First, let us create a new Spark pool with version 3.0 in Azure Synapse: One of the most significant improvements in Spark 3.0 is Adaptive Query Execution.…
Helm charts provide a powerful way to package and deploy applications to AKS with Azure specific integrations. By leveraging values files for environment…
Azure Container Apps combines the simplicity of serverless with the power of containers. With built in Dapr integration, KEDA scaling, and revision…
Dapr simplifies building cloud native applications by providing consistent APIs for common microservices patterns. With native Azure integration, you can…
Azure Service Fabric provides a mature platform for building microservices with built in state management, service discovery, and health monitoring. While…
The Circuit Breaker pattern is essential for building resilient microservices. Polly provides a mature implementation with advanced features like sampling…
The Saga pattern is how you handle distributed transactions—operations that span multiple microservices and need consistency guarantees—without two-phase…
Event Sourcing with Azure Cosmos DB provides a robust foundation for building systems that require complete audit trails, temporal queries, and event driven…
CQRS with Azure services enables building highly scalable applications with optimized read and write paths. Azure SQL provides transactional guarantees for…
The Azure Architecture Center is the reference resource I bookmark more than any other when I need to move from "I know what I want to build" to "I know the…
The Azure Well-Architected Framework is the document I send to clients at the start of every architecture engagement and the checklist I review before every…
Azure Advisor is the free recommendation service that surfaces findings your team should have already found—and usually hasn't. The five categories (Cost…
Effective cost management requires proactive monitoring and automated responses. Azure Cost Management budgets provide the foundation for cost governance,…
Multi-tenant Azure management is the architecture problem I've spent more time on this year than any other. The patterns matter because the wrong choice…
Azure Lighthouse transforms multi tenant management from a fragmented experience to a unified operational model. Whether you're a managed service provider…
Azure Management Groups are the first thing I help clients set up when their Azure estate starts involving more than two or three subscriptions. Without…
Azure Resource Graph is the query service that answers "what do we actually have in Azure?" at enterprise scale in milliseconds. The ARM APIs query each…
ARM Template Specs provide a governed, versioned approach to managing infrastructure templates. With native RBAC integration and first class Azure resource…
Bicep modules enable clean, maintainable Azure infrastructure code. By separating concerns into modules, you create reusable components that can be shared…
Pulumi is the infrastructure-as-code choice for teams where "I don't want to learn another DSL" is a genuine constraint. You write infrastructure in C#…
Terraform with the AzureRM provider is the infrastructure-as-code choice I recommend to teams that need multi-cloud portability or already have Terraform…
GitHub Actions workflows for Azure deployment have become my default for any project that lives on GitHub and targets Azure. The Azure/login action with…
.NET 6 Preview 4 in June 2021 was the point where I started genuinely excited about the November GA. Minimal APIs, Blazor improvements, native AOT…
ASP.NET Core on Azure App Service is the combination I've deployed more than any other in the past three years, and the relationship between the .NET…
Let's start with a new Blazor WebAssembly project: A typical Blazor WASM project includes: The most straightforward option for client side Blazor apps:…
Subdomains are the simplest to configure. You'll create a CNAME record pointing to your Static Web App's default hostname.
Application Gateway v2 is where the regional Layer 7 load balancer story became production-serious for enterprise workloads. Autoscaling means you don't…
Azure Front Door Standard/Premium is the convergence I've been waiting for since the days when you had to choose between classic Front Door (global routing…
Platform Use Case Global/Regional Application Gateway Regional apps, traditional Regional Front Door Global apps, edge protection Global CDN Static content…
Basic (Free) Always on traffic monitoring Automatic mitigation Protection for Azure infrastructure No SLA or customization Standard (Paid) All Basic…
The headline announcement for me was the integration of GPT-3 into Power Apps. You can now describe what you want in plain English, and GPT-3 generates the…
Feature Key Vault Premium Managed HSM HSM Type Multi tenant Single tenant FIPS Level 140 2 Level 2 140 2 Level 3 Admin Control Azure managed Customer…
Key Vault stores certificates as: Certificate : The X.509 certificate (public) Key : The private key (protected) Secret : The combined certificate + private…
Flow Use Case User Interaction Authorization Code Web apps, mobile Yes Authorization Code + PKCE SPAs, mobile, desktop Yes Client Credentials Daemon/service…
MSAL handles: OAuth 2.0 protocol flows Token caching Token refresh Multi account support Conditional Access handling For APIs calling downstream APIs: MSAL…
Azure AD : Identity provider OAuth 2.0 & OpenID Connect : Protocols Microsoft Authentication Library (MSAL) : Client libraries Microsoft Graph : API for…
Guests appear in your directory but authenticate elsewhere.
User Flows: Pre-built, configurable through portal Custom Policies: XML-based, fully customizable
Conditional Access is where the "never trust, always verify" principle of Zero Trust actually gets operationalised. Every sign-in to every app goes through…
The service identifies: Reconnaissance : Attackers gathering information about your environment Compromised credentials : Pass the hash, pass the ticket,…
Microsoft Defender for Cloud includes: CSPM (Cloud Security Posture Management): Free tier with security recommendations CWP (Cloud Workload Protection):…
In Sentinel, SOAR is implemented through Playbooks (Logic Apps) and Automation Rules.
Container Insights is the built in monitoring solution for AKS: This deploys: OMS agent as DaemonSet Metrics collection Log forwarding to Log Analytics…
Azure Monitor for Prometheus addresses these while maintaining compatibility.
Grafana on Azure became substantially easier when Azure Managed Grafana reached general availability. Before that, teams ran Grafana on AKS or a VM, managed…
For development or small workloads: For production workloads, use AKS with persistent volumes:
TimescaleDB is the time-series database I recommend to teams that are already comfortable with PostgreSQL and don't want to learn a new query language or…
Hyperscale uses the Citus extension to create a distributed database: Coordinator : Routes queries, stores metadata Workers : Store data shards, execute…
For larger databases, use Azure Database Migration Service.
Azure SQL Managed Instance is the SQL Server target I recommend most for lift-and-shift migrations from on-premises SQL Server. It's PaaS—no OS to patch…
Arc data services let you run Azure-managed databases on your own infrastructure while using Azure tools for management.
The Cosmos DB integrated cache is the feature I've been waiting for since the first time I watched a read-heavy application burn through its RU budget…
Cognitive Services in mid-2021 is a sprawling catalogue—vision, speech, language, decision—that can be disorienting to navigate. Today's post is the…
The goal is reducing the time from idea to edge-deployed AI from months to days. The platform uses Azure Custom Vision under the hood but abstracts away the…
Graph Data Connect solves this by delivering bulk extracts of Microsoft 365 data directly to Azure Data Factory.
Power Platform governance is the problem that doesn't announce itself until it's already serious. I've walked into tenants where three hundred flows were…
Applied AI Services are Microsoft's answer to "I need AI in this process, but I don't want to wire up five Cognitive Services and build the integration…
SQL Server to Azure SQL migration is the database journey I've been on more times than any other, and the first decision that shapes everything else is…
1. Assess compatibility first Use DMA for SQL Server 2. Test migrations in non prod Validate before production 3. Plan for downtime Even online migrations…
Azure Migrate is the tool that turns a migration conversation from "we think we have about 300 servers" to "we have 287 servers, here's the dependency map…
Private Cloud Dedicated VMware environment Clusters Groups of ESXi hosts (minimum 3) vSAN Software defined storage NSX T Software defined networking HCX…
Azure Dedicated Host is the offering I typically reach for when a client's security or compliance team says "I can't be on shared hardware." The use cases…
Confidential Computing addresses the attack surface that most cloud encryption doesn't touch: data while it's actively being processed in memory. Encryption…
Responsible AI went from a philosophy discussion to a practical engineering requirement in a short time. The catalysts I've seen in real projects: a loan…
Azure ML Designer is the no-code visual pipeline builder for ML, and its place in the toolbox is narrower than the marketing suggests. It's genuinely useful…
AutoML automates: Feature engineering Automatic featurization Algorithm selection Tests multiple algorithms Hyperparameter tuning Optimizes model parameters…
The Synapse Pipelines vs. ADF question comes up in almost every Synapse project I start. The honest answer depends on where the rest of your data platform…
The Self-Hosted Integration Runtime is the component that makes ADF viable for the majority of real enterprise projects—the ones where the source database…
ADF trigger design is where data pipeline reliability is won or lost. Schedule triggers are obvious—run at midnight, run every hour—but the interesting…
Both work together - a user needs both RBAC permission to access the storage account AND appropriate ACL permissions on the specific path.
ADLS Gen2 is the storage layer that most enterprise data platforms in Australia are converging on. The combination of blob-scale storage (petabytes, low…
Azure SQL to Cognitive Search is a combination I've set up half a dozen times, and the reason it keeps appearing is that most enterprise organisations…
When indexing blobs, the indexer performs several operations: 1. Enumerate blobs Find blobs matching your criteria 2. Download content Retrieve blob data 3.…
Indexers handle the ETL process for search: Requires a column that increases monotonically: Enable change tracking on your database: Source JSON: Source…
Custom skills are Azure Functions that implement a specific contract.
Logic Apps connectors are the product feature that makes integration architects either love or dread the platform. Over 400 connectors covering everything…
Durable Entities are addressable units of state that process operations one at a time. Think of them as lightweight actors - each entity has a unique…
The magic happens through the Durable Task Framework, which handles state persistence, checkpointing, and replay automatically.
Anomaly Detector is the Cognitive Service I pull out when someone asks "can you tell us when something goes wrong?" and the answer is "I don't know exactly…
Form Recognizer is where I've seen the most immediate ROI from Cognitive Services in enterprise settings. Accounts payable teams manually keying invoice…
Custom Vision fills the gap that the general-purpose Computer Vision API can't: domain-specific image classification and object detection where the model…
Computer Vision is the Cognitive Service that covers the "I have an image and I need to know what's in it" scenario without training a custom model. Read…
Azure Translator is one of the Cognitive Services that I've shipped most quietly—it tends to be a two-day integration rather than a headline feature, but…
Intents : Categories of user actions (e.g., BookFlight, GetWeather) Entities : Important data to extract (e.g., locations, dates, quantities) Utterances :…
QnA Maker simplifies building knowledge based chatbots: Easy content import from URLs, documents, and manual entry Natural language understanding for…
I've built bot projects with the Bot Framework that I'm proud of, and bot projects that I wish I could forget. The difference is almost always scope…
Azure Speech Services enable rich voice experiences: Speech to Text : Real time transcription with high accuracy Text to Speech : Natural sounding neural…
ThreadX : RTOS kernel with preemptive scheduling NetX Duo : TCP/IP network stack (IPv4 and IPv6) FileX : FAT compatible embedded file system GUIX : Graphics…
1. Azure Sphere MCU : Secured silicon with hardware based security 2. Azure Sphere OS : Linux based secured operating system 3. Azure Sphere Security…
Azure Digital Twins was the service that reframed how I think about IoT data. Most IoT architectures I'd built treated device telemetry as a stream: events…
Device twins are the feature in Azure IoT Hub that solved a problem I didn't fully appreciate until a client had 3,000 industrial sensors in two factories…
Event Hubs with Kafka protocol offers: No Kafka cluster management Automatic scaling 99.99% SLA Native Azure integration Pay per throughput pricing Event…
Azure Stream Analytics sits in a specific niche that I've come to appreciate: SQL-native stream processing for teams who shouldn't have to learn Spark or…
The appeal of Spark Structured Streaming is that you write it almost identically to a batch Spark job. Same DataFrame API, same transformations, same Spark…
Delta Lake time travel is the feature that makes "oops" survivable. An engineer runs a DELETE with a typo in the WHERE clause. A batch job writes corrupt…
Databricks workspace governance was the problem nobody thought about until there were thirty workspaces, fifteen clusters running overnight, and six teams…
A semantic model sits between raw data and end user tools, providing: Business friendly names and descriptions Calculated measures and KPIs Row level…
Paginated reports excel when you need: Pixel perfect formatting for print Multi page documents with headers/footers Parameter driven report generation…
Power BI Premium Per User launched as the answer to "my team needs paginated reports, deployment pipelines, and large dataset support but we don't have a…
Data flows in Synapse run on Spark clusters and offer: Visual drag and drop interface 80+ built in transformations Schema drift handling Data preview and…
Dedicated SQL pools distribute data across 60 distributions for parallel processing: Control node : Orchestrates queries and manages metadata Compute nodes…
The question that sparked this post came from a data engineer on a client project: "can we run Power BI reports against Cosmos DB without destroying the…
We had a Cosmos DB Serverless conversation at the end of 2020 and I'm returning to it because the experience of using it in actual development over two…
Azure Web PubSub is the lower-level real-time service that sits alongside SignalR. Where SignalR abstracts the transport and provides a hub model for your…
I've tried to roll my own WebSocket fanout exactly once. It worked beautifully for 200 concurrent connections and melted at 2,000. Azure SignalR Service…
Traditional configuration approaches using appsettings.json or environment variables have limitations: No central management Requires redeployment for…
ACI is the container service I reach for when I need a container running in under two minutes and I don't want to explain what a node pool is. Per-second…
Many enterprises have legacy .NET Framework applications that cannot easily migrate to .NET Core. Windows containers provide a path to containerization…
AKS networking is the part of Kubernetes that looks like a detail until it becomes a blocker. "I can't reach the on-prem database from the cluster" is a…
Azure Arc is the product I've recommended most in 2021 that clients are slowest to adopt. Not because they don't need it—they do—but because "manage your…
Azure Hybrid Benefit is the discount that nobody in finance remembers to claim and every consultant I know flags in the first cost review. If you have…
Reservations are the Azure cost conversation nobody has early enough. Pay-as-you-go is convenient; reserved instances are a commitment that rewards the…
Cloud bills have a way of arriving late and being everyone's problem at once. Cost Management is the tool that moves that conversation from quarterly shock…
Application Insights is the monitoring tool that pays you back in proportion to how much you invest in it. Out of the box you get…
KQL is the query language I've spent the most hours in this year that most developers have never heard of. If you use Log Analytics, Sentinel, Application…
Azure Monitor supports several alert types: Metric alerts Based on metric values crossing thresholds Log alerts Based on Log Analytics query results…
I've written a lot of ARM templates in the past three years, and I've also inherited a lot of ARM templates written by people who were having a bad day. The…
GitHub Actions went from "interesting alternative to Azure DevOps Pipelines" to "my default CI/CD choice for most new Azure projects" in about twelve…
Run the task: 1. Use Premium SKU for production with geo replication and security features 2. Enable content trust for image signing 3. Implement lifecycle…
Batch is underused. Every time I describe it to an engineering team—"managed pool of VMs, automatic scaling, job and task model, retry on failure…
The first ML pipeline I helped build was a series of Python scripts duct-taped together with a bash wrapper and a nightly cron job. It worked exactly as…
HDInsight supports multiple cluster types for different workloads: Apache Spark Fast, general purpose cluster computing Apache Hadoop Batch processing with…
Synapse Spark pools are the feature that stops the "should I use Databricks or Synapse?" question from being purely a product choice and makes it an…
Mapping Data Flows are the feature that ended most of my "should I use PySpark or Spark SQL for this?" debates. Visual, yes—but reviewable in JSON and…
Azure Firewall Manager provides: Hierarchical policy management Parent and child policies for inheritance Global deployment Manage firewalls across regions…
Virtual WAN simplifies enterprise networking by providing: Hub and spoke architecture Automated connectivity between hubs and spokes Branch connectivity VPN…
I keep writing about Bastion because I keep finding jump boxes I have to replace. VM with a public IP, port 3389 open, "admin" password in a sticky note in…
Private Link provides several key benefits: Private connectivity Access services over private IP addresses Data exfiltration protection Service endpoints…
Front Door's global load balancing and caching are easy to understand. The Rules Engine is the part that turns it into a real edge platform. Conditions on…
MLflow is the closest thing to a standard the ML tooling space has right now—experiment tracking, run metadata, model registry, and a deployment abstraction…
The first time I helped a platform team stand up a new environment from scratch, we got it right on the third try. The first two tries had a RBAC assignment…
The tagging policy conversation is the one I have in every landing zone engagement. Someone inevitably says "we'll just ask people to tag resources…
Azure Defender extends the capabilities of Azure Security Center by adding: Advanced threat detection using machine learning and behavioral analytics…
Most recommendation systems I've seen in enterprise settings are either "sort by recency" in a trench coat, or a Spark job that runs weekly and calls itself…
Flexible Server is the deployment option I now default to when a client asks "which PostgreSQL on Azure should I use?" The original Single Server mode…
If you're running Spring Boot applications, you've probably dealt with the complexity of setting up Kubernetes clusters, configuring service meshes, and…
Traffic Manager keeps showing up in my designs because it's cheap, simple, and works at the layer most failover stories actually need: DNS. The catch is…
Disaster recovery is the topic everyone agrees is important and nobody wants to be assigned. ASR is the service that turns "we should test my DR" from a…
Backup configurations have a way of looking fine until the day you need to restore. Default policies, retention that doesn't match the actual RPO…
"Run analytics on Cosmos data without burning RU/s on the OLTP container" used to mean ETL-by-night. Synapse Link makes the trade-off go away. A…
Default Cognitive Search will get you to "decent enough." Excellent search is a tuning exercise, and the levers that matter are mostly hidden. Custom…
Cosmos DB Change Feed is the feature most teams discover six months after they've already hand-rolled a polling loop they'll regret. It's a sorted…
"Put it in Key Vault" is the easy advice. The advanced game is rotation, lifecycle, and access patterns that don't break under pressure. Managed identity…
I revisit APIM policies on this blog roughly every six months because the patterns are how the service earns its keep. Rate limiting per subscription, JWT…
"Where is the money going?" is the most expensive question in cloud. Azure Advisor is the free service that answers a surprising amount of it for you.…
Operational dashboards in Azure used to mean Power BI, a custom data export, and a refresh schedule nobody could remember. Workbooks killed all of that for…
Most enterprises I work with have Kubernetes in three places they didn't plan for: an on-prem cluster the platform team built, an EKS estate from an…
An accounts payable team I worked with last quarter was processing 4,000 invoices a month by hand—open the PDF, retype line items into the ERP, repeat. The…
Azure Communication Services launched late last year, and "build your own video calling app on the same backbone as Teams" is the pitch that makes it click…
Static Web Apps shipped its preview late 2020 and has been quietly stealing front-end workloads from App Service ever since. Today I want to dig into the…
If you've used Log Analytics or Sentinel, you've already used Azure Data Explorer—they're built on the same engine. ADX as a standalone product is the thing…
Service Bus is the message broker I most often pull off the shelf in 2021, and it's the advanced patterns—not the basic queue—that earn it the spot.…
Service Bus is fine for service-to-service messaging, but when you want "anything in the system can listen for the order created event," Service Bus topics…
Logic Apps Consumption was great for "wire up two SaaS apps quickly" and increasingly painful at scale—shared multi-tenant runtime, no VNet integration…
Cosmos DB Serverless: scale-to-zero for your NoSQL workloads.
SQL Analytics is Databricks' bid for the BI persona who never wanted to learn PySpark. A T-SQL endpoint over Delta Lake, a query editor that looks like…
Artificial Intelligence (AI) and Machine Learning (ML) are trending topics right now. In 2021, there are countless of ways to have a form of "AI" in your…
"The model said no, and I can't tell why" is the conversation that derails most ML deployments. Responsible AI in Azure ML is a set of tools designed to…
With containers becoming the standard deployment unit for modern applications, AKS provides: Managed control plane (no cluster management overhead)…
ARM templates and I have a long, unhealthy relationship. Hundreds of lines of JSON to express what reads like fifteen lines of intent. Bicep is the cure…
Serverless SQL: query anything, pay for what you scan.
.NET 5 is out, and Functions has a new hosting model in flight: isolated process. Instead of running inside the Functions host, your function runs in its…
SQL Ledger: trust through cryptographic verification.
First post of 2021. The kids are still on summer holidays here in Australia, the home office is half-disassembled for cleaning, and I'm sketching out what I…
It's New Year's Eve, the kids are asleep, and I'm wrapping up a year that nobody planned for. 2020 was the year cloud stopped being a strategic conversation…
Storage accounts are deceptively easy to deploy and easy to misconfigure into a leak. Public blob containers are still the most common "how did this get…
Spinning up a Databricks workspace is the easy part. Making it production-ready is where I see the most avoidable rework. Cluster policies, AAD passthrough…
Log Analytics is the database I've spent the most hours in this year that nobody calls a database. It's where every Azure diagnostic, AppInsights trace…
App Gateway v1 was a credible Layer 7 load balancer. v2 is the upgrade that makes it the default I reach for in front of internal AKS clusters and App…
LUIS: teaching machines to understand human language.
Christmas Day, and a quiet thought from a quiet house: most "IoT projects" never get past slide three because the team doesn't have the platform to back the…
"Can you log in to my tenant and fix it?" used to mean a guest account, an awkward password share, and a privileged role for a partner that frequently…
Every migration project I've run has the same first two weeks: spreadsheet hell. What's running, what does it talk to, what's the dependency graph, what…
DDoS Protection Standard provides cost credits for attack-related scaling. Azure DDoS Protection: resilience against the largest attacks.
Jump boxes are one of those legacy patterns I keep finding in environments I inherit. A Windows VM with RDP open to the internet, "for admin access only,"…
Azure Communication Services: Teams-grade communication in your apps.
DSC: configuration as code for infrastructure compliance.
Azure NetApp Files: enterprise storage performance in the cloud.
Azure Media Services: broadcast-quality video for everyone.
Push notifications look easy until you're maintaining four sets of credentials (APNs, FCM, WNS, ADM) and writing platform-specific payloads for each.…
Azure Spring Cloud: enterprise Java without infrastructure burden.
Cosmos DB Serverless: NoSQL without the commitment.
"Why is the staging environment talking to the production database?" is a sentence I never want to hear again, and yet—when configuration lives in…
Video Indexer: unlock the content inside your videos.
Functions has supported "any language" via the worker model for a while, but custom handlers are the cleaner option when your runtime isn't on the supported…
Every dashboard request that hit my inbox in 2019 had the same wishlist: parameters at the top, charts that talk to each other, KQL on the inside, and…
Azure Firewall: cloud-native network security without the complexity.
The hub-and-spoke topology I've drawn on whiteboards for years is what Virtual WAN turns into a deployable resource. A managed hub per region, automatic…
The first ML model I helped put into production was a notebook a data scientist ran by hand every Monday morning. That worked exactly as well as you'd…
"It can't be processed if it can't be decrypted, but you can't process encrypted data." That truism quietly stopped being true. Confidential Computing uses…
All of this is unified through Synapse Studio - a single workspace where data engineers, data scientists, and analysts can collaborate.
"Can I use Cognitive Services if my data can never leave the country?" I get this question from public-sector and healthcare clients constantly. The answer…
A WAF in front of every public app is no longer a nice-to-have. Bots scan you within minutes of going live, and "we'll add WAF later" is how breaches…
Embarrassingly parallel workloads are the ones I most enjoy moving to Azure. Render farms, Monte Carlo simulations, batch image processing, genomic…
DR planning is the part of every project that gets bumped to "Sprint 9" and then to "next quarter." ASR is good enough that it makes the bump indefensible.…
Every backup conversation I've ever had eventually circles back to a story about someone restoring a tape only to discover it was empty. Azure Backup is the…
Logic Apps is the first thing I build in when a client says "we need to integrate with Salesforce, then push to SAP, then notify the team in Teams."…
ExpressRoute: enterprise-grade private connectivity to Azure.
Every consultancy I've worked at has the same problem: developers need a sandbox, finance needs to not get a heart attack, and three months later there are…
Traffic Manager is a deeply unglamorous service that has saved a couple of my projects. It works at the DNS layer—no traffic flows through it, just the…
Most location features I've shipped lived on Google Maps for one reason: nobody knew Azure had a maps service. Azure Maps is competitive now—routing…
Streaming every sensor reading to the cloud sounds clean until you cost the bandwidth, or until the 4G link in the back of a truck drops for an hour. IoT…
There's a moment in any growing SaaS where someone runs spspaceused and the room goes quiet—the database is approaching the 4 TB ceiling and the migration…
I have a love-hate relationship with visual ETL tools. Drag-and-drop is wonderful right up until you need to diff two pipelines in source control. ADF Data…
SIEM used to mean a rack of appliances, a forwarder per data source, and a six-figure annual licence. Sentinel rewrites that economics. It's a SIEM that…
The first time a Private Endpoint refused to resolve correctly because the VNet was using Azure-provided DNS, I learned this lesson the hard way: private…
Deploy JMeter on Azure VMs for distributed testing: Run JMeter in containers for quick, scalable tests: Metric Description Target Response Time (avg)…
API Management is one of those services I underestimated for years. "It's a proxy, right?" The penny drops when you realise policies are the product. Rate…
Indexing PDFs is easy. Indexing PDFs in a way that makes them findable is a different sport. Cognitive Search skillsets are the part I usually sell to…
Most enterprises I work with aren't "going to the cloud"—they're running half their estate on-prem, a slice on AWS, a dev environment on a colleague's…
"We have data and we want machine learning." I get this conversation often, and most of the time the team doesn't need a data scientist on day one—they need…
Service Fabric: enterprise microservices with built-in state management.
A general-purpose database can store time-stamped data. It just gets sad about it once you're past a few hundred million rows and someone wants a five-year…
Azure Defender: security visibility across your entire estate.
Every cloud bill story I've heard starts the same way: "we didn't realise it was running." A test VM left on over a long weekend, a Premium SSD attached to…
The first Azure Function project I shipped used static everything. Static config, static SQL helpers, static logger. It worked—until I tried to write a unit…
Data Box: when the network isn't fast enough, ship it.
Notebooks are great until you need them to run on Tuesday at 2am, retry on failure, and chain into the next step. That's where Databricks Jobs come in.…
Cold starts on Consumption plan kill credibility. I've watched a perfectly good HTTP-triggered function get blamed for "the API being slow" because the…
A facility manager I worked with last year asked the same question every IoT pitch eventually triggers: "great, you can stream sensor data—but can you tell…
Microsoft announced Azure Purview at Ignite this week and the timing could not be better. Every consulting engagement I've been on this year has the same…
A second pass at ACR, this time with the lessons I've collected from running it for clients. Use the geo-replicated SKU only if you actually need…
A second pass at Form Recognizer, focused on what's changed since I last wrote about it. The prebuilt invoice and receipt models keep getting better — the…
Static Web Apps with the integrated Functions backend is the deployment story I keep recommending for small SaaS dashboards and internal tools. Static…
Azure Bot Service bridges AI and conversation.
Note: While dedicated SQL pools are mature (evolved from Azure SQL DW), the Synapse unified experience is still in preview.
Redis caching transforms application performance at scale.
Best practice: use Managed Identity where supported. Linked Services are the foundation of Data Factory connectivity.
Application Insights is essential for production observability.
Bindings let you focus on business logic, not plumbing.
Accept webhooks from any source. Create connectors for your APIs. Logic Apps is the integration glue for enterprise systems.
Event Grid enables reactive, event-driven architectures at scale.
Encrypt sensitive columns—keys never exposed to SQL Server. Azure SQL security is comprehensive—use all layers for maximum protection.
Microsoft Ignite 2020 was different this year - a fully virtual 48-hour event instead of the usual 5-day in-person conference. Despite the condensed format…
Share VPN/ExpressRoute gateway across peered VNets. VNet peering is the foundation of Azure network architecture.
Of every Cosmos DB design conversation I've had, partition key choice is the one that has the biggest cost-and-performance consequence and the smallest…
Function chaining, fan-out/fan-in, human interaction, and eternal orchestrations — Durable Functions exposes a small set of patterns that cover most…
Azure DevOps Pipelines was the obvious answer for years. GitHub Actions in 2020 changed that calculus — for any project where the source already lives on…
B2C handles the complexity of customer identity so you can focus on your application.
Azure Kubernetes Service: Managed Kubernetes
"We want AI in my app" is a sentence I hear at least twice a month. Eight times out of ten the answer isn't "train a custom model" — it's "use Cognitive…
Resource Graph is essential for understanding your Azure estate at scale.
Blueprints ensure every environment starts from a known, compliant state.
Every Azure environment I've inherited from someone else has the same three problems: storage accounts created without encryption-at-rest configuration…
A second Synapse-preview note, on what I think is the genuinely new idea in the platform: serverless SQL. Point T-SQL at parquet, CSV, or JSON in your data…
Spark on Synapse is the same managed Spark story most cloud providers offer now, but it's done with the parts of Azure I already use — same workspace as the…
Gen2 is the standard for new data lakes on Azure. ADLS Gen2 is the foundation. Build your data platform on solid ground.
Stream Analytics is one of those services I keep recommending and clients keep being surprised by — SQL on top of an event firehose, with windowing and…
Synapse Link: analytics without the pipeline complexity.
Synology reached out to me to give their new DiskStation DS920+ a review. If you're not yet aware of Synology, they are the leading NAS…
.NET on Functions is the default story Microsoft tells, but in practice the serverless workload that lands on my desk most often is a Python data scientist…
Savings: $1,700/year for 1TB Set up lifecycle policies on day one. Future you will thank past you for the cost savings.
Every cloud team eventually accumulates a folder full of "scripts I run monthly" — clean up old resource groups, rotate certificates, scale down dev…
"Why don't I just use Azure Load Balancer?" is the question I get most often when someone first sees Application Gateway in the architecture diagram. The…
Strong and Bounded Staleness cost double because reads wait for quorum. Start with Session - it provides intuitive consistency for users while maintaining…
Elastic pools are ideal for SaaS multi-tenancy where tenant databases have varied, unpredictable loads.
Service Bus is the backbone of reliable, decoupled architectures on Azure.
MLflow makes ML experiments reproducible and models traceable.
Front Door is the global entry point for serious production workloads.
A Data Factory pipeline without a trigger is a stored procedure that nobody calls. Triggers are the boring half of orchestration that decides whether your…
The extra cost of Bastion is worth the security posture improvement.
Alerts should be actionable. If you can't do anything about it, it's noise.
By default, Azure PaaS services (Storage, SQL, Cosmos DB) have public endpoints. Even with firewall rules, data transits the public internet.
For "I need to run this container for an hour to crunch some data," ACI is the right answer roughly nine times out of ten. AKS is overkill, App Service for…
Three years ago a client asked me to "put a gateway in front of my APIs." It turned into a six-month conversation about rate limiting, OAuth, partner…
Things you can't do in Azure SQL Database but can in Managed Instance. Real-time replication from on-premises SQL Server to Managed Instance - useful for…
Redis Enterprise: when standard caching isn't enough.
Logic Apps excels at the unglamorous middleware work — turning the SOAP service the warehouse system exposes into the REST endpoint the new mobile app…
Designer democratizes ML for teams that don't live in code.
The Kafka API compatibility makes Event Hubs a drop-in replacement for many streaming scenarios.
Anyone who's run a "data lake" for any length of time has hit the same wall: parquet files everywhere, no transactional guarantees, partial-write disasters…
The Azure DevOps Classic editor is comfortable, and I've watched many teams resist moving away from it for that exact reason. Until they need to branch a…
After several years of writing ARM templates, picking up Terraform felt like getting glasses for the first time. The same infrastructure-as-code idea, but…
Most "search" tutorials I see stop at "create an index, push some JSON, query it." That's the easy 10%. The 90% that matters in real projects is the indexer…
For log analytics at scale, ADX is hard to beat.
Change Feed transforms Cosmos DB from a database into a real-time event platform.
Durable Entities offer a middle ground. State is automatically persisted and scales across multiple instances. For our inventory tracking, response times…
After committing to writing again, I needed to ensure my blog infrastructure was solid. Azure Static Web Apps, announced at Build 2020 in May, is currently…
Durable Functions provide the reliability and coordination capabilities needed for complex serverless workflows, while maintaining the cost benefits of…
"Customers ask us the same five questions over and over." Every support team I've worked with has said this. Before LLMs ate the world, QnA Maker was the…
"We have a data warehouse and a Hadoop cluster and they don't talk to each other" was a sentence I heard for years. Synapse is Microsoft's answer: a single…
Azure Container Registry provides a secure, scalable foundation for container-based deployments on Azure.
Most Logic Apps content online is about Office 365 connectors and approval workflows — and that's fine, but the enterprise use case I keep running into is…
Azure Cognitive Search provides powerful search capabilities that can transform how users find and discover content in your applications.
There's a recurring confusion among teams new to Azure: which messaging service do I use? Service Bus, Event Hubs, Event Grid, Storage Queues — they all…
Every accounts payable team I've ever talked to has a person whose job is, in part, retyping data from PDF invoices into an ERP. It's exhausting work, the…
In the first six months of the pandemic, "we need everyone working from home by Monday" became a sentence I heard from clients more times than I can count.…
A surprising number of "modernise my reporting" engagements I see start the same way: data scattered across an on-prem SQL Server, a SaaS CRM, three Excel…
Two services that worked perfectly in isolation, plus an HTTP call between them, equals a system that occasionally drops orders. Every consultant has been…
A confession to start: I've shipped connection strings in appsettings.json more times than I'm proud of. The reasons are always the same — "it's only dev"…
First thing I do on any new Azure-hosted application I'm asked to look at: open Application Insights and see what's actually happening. Half the time, "the…
SaaS clients with per-tenant database isolation hit the same wall once they cross about 20 customers: the bill for a hundred half-idle Standard tier…
Storage bills creep. They never spike, they never alert, they just slowly become a line item somebody at finance asks about, and by then you've got several…
"How do we deploy a new version without downtime?" That's the conversation that got me writing this post. AKS gives you the primitives, but the choice…
A retail client this week handed me six months of customer feedback in a CSV and asked the question I get every couple of months: "what are people actually…
A client this month wanted dashboards living inside their existing customer portal — "looks like my app, not like Power BI." That's the embedded analytics…
SDK v3 has been out for about a year now and I've finally migrated all the Cosmos work I had on v2. The migration was less painful than I expected — the API…
A lot of the work landing on my desk this year has been "we suddenly need this in the cloud, and we needed it last week." Azure Functions on .NET Core 3.1…