8 min read
GPT-6 Landed in Foundry: Upgrade Discipline Beats Upgrade Speed
GPT-6 arrived in Foundry ten weeks after GPT-5.6. Pin versions, gate on real queries, shadow traffic and plan retirements before you switch tiers on day one.
Azure OpenAI
142 articles
9 min read
GPT-5.6 Sol, Terra or Luna: Pick the Tier per Job, Not per App
GPT-5.6 in Foundry has a 5x price spread across Sol, Terra and Luna. Choose a tier for each workflow step, back it with eval gates, then pick deployment types.
8 min read
Why an LLM Response Cache Moves Your p50 but Rarely Your p95
A response cache only speeds up the requests it hits, so the tail stays slow until hit rates get very high. Here's the arithmetic and what fixes p95 instead.
7 min read
Output Contracts: Catch a Model Swap Before Downstream Systems Do
Structured outputs fix the shape of a model's answer, not its meaning: a two-layer contract between a Foundry model and the systems that consume it.
8 min read
Splitting an AI Journey's Latency Budget Across Its Steps
A journey-level latency budget only works when each step gets a cap, a timeout and a fallback; here is how to split one across a RAG request.
9 min read
RAG Citations People Can Check: Treat Them as Part of the Answer
Good retrieval is wasted if users cannot verify the answer, so design RAG citations to be checkable, validated in code and measured like relevance.
8 min read
Canary a Model Swap in Foundry: Sticky Splits and Version Stamps
An offline evaluation can't tell you how a new model handles real traffic. How I'd canary a Foundry model change with sticky splits and per-call version logs.
8 min read
Four LLM Caches and How to Tell Which One Will Pay Off
Prompt, response, semantic and retrieval caches save different money and add different risks. Here's a break-even check to run before you build one.
7 min read
Latency Budgets as a Cost Control: Pricing Each AI User Journey
Set a latency budget per user journey, then let it choose the Azure OpenAI deployment tier, model and reasoning effort you actually pay for.
9 min read
Model Retirements in Foundry: Choose the Upgrade Policy on Purpose
Every Foundry model deployment has a version upgrade policy, and the default swaps your model at retirement. How to choose, audit and run model changes safely.
8 min read
Where LLM Latency Goes, and Which Token Cuts Actually Help
Input and output tokens cost and delay answers differently, so split latency into first-token and generation time before trimming anything.
7 min read
A Token Waste Audit: Cut What the Answer Doesn't Use
Profile each LLM request by segment, rank the cuts by quality risk, and keep only the token savings that survive a paired evaluation against the baseline.
8 min read
Fine-Tuning or RAG? Decide by the Failure You're Fixing
Fine-tuning and RAG fix different failures. How to tell which one you have on Microsoft Foundry in early 2026, what each costs, and when to combine them.
8 min read
Why Your Azure OpenAI Prompt Cache Keeps Missing
Azure OpenAI caches prompt prefixes automatically, yet many apps see few cache hits. How to measure the hit rate and fix the prompt shapes that break it.
6 min read
Semantic Kernel Apps in February 2026: Stay, Bridge, or Rewrite
Agent Framework is still in preview and Semantic Kernel is in fix-and-stabilise mode: how to decide whether to stay, bridge, or rewrite in February 2026.
8 min read
Monitoring AI in Production: Layers, Owners and a Review Rhythm
Run AI monitoring as an operating rhythm: four layers with owners, what pages versus what waits, and tagging every change so regressions are traceable.
8 min read
LLM-as-a-Judge: Check the Judge Before You Trust the Score
An LLM judge has its own biases: how to write pass/fail criteria, measure its kappa and false-pass rate against human labels, and know when to skip it.
8 min read
Azure OpenAI vs OpenAI in 2026: Same Code, Different Contract
With the Azure OpenAI v1 API one OpenAI client works on both, so the choice rests on data processing location, identity, networking, capacity and model timing.
8 min read
Learning AI Without a PhD: What Transfers From Cloud Engineering
How I moved from cloud and data engineering into AI work without a maths degree: what I learnt in which order, what I skipped, and where engineers go wrong.
8 min read
AI Security Basics: Keys, Quotas and Logs Before Prompt Tricks
The AI security failures that actually cost money are leaked keys, unbounded token spend and sensitive data in logs. Fix those before prompt defences.
8 min read
Cost Per Query: Measure the Unit Cost Before You Cut It
Treat cost per query as the unit metric for an LLM app: decompose it from the usage object, then cut it in order with model choice, caching, context and batch.
8 min read
Semantic Kernel Agents in Production: What's Stable, What Isn't
Running Semantic Kernel agents in production in early 2026: which APIs are GA, which are preview, how to guard tool calls, and where Agent Framework fits.
7 min read
Eight Prompt Engineering Myths to Drop in 2026
Eight prompt engineering myths worth dropping in 2026, from verbose prompts to reflexive chain-of-thought and temperature rules, and what to measure instead.
8 min read
Azure OpenAI Hidden Costs: What Your Token Estimate Misses
Re-sent history, hidden reasoning tokens, billed failures, idle fine-tuned deployments and re-embedding: where Azure OpenAI bills drift from the estimate.
1 min read
Building a Custom GPT for Your Documentation with Azure OpenAI
This pattern works for any documentation. The key is good chunking, quality embeddings, and a well-crafted system prompt that keeps the bot focused on your…
1 min read
Building Multi-Agent Systems with AutoGen and Azure
Instead of a single LLM handling all tasks, multi-agent systems divide work among specialized agents that communicate and coordinate to achieve goals.
1 min read
Model Fine-Tuning on Azure: When and How to Customize LLMs
Fine-tuning makes sense when you need consistent style, specialized terminology, or improved performance on specific tasks that prompt engineering cannot…
1 min read
Semantic Kernel: Orchestrating AI Agents with Plugins and Planners
Semantic Kernel abstracts the complexity of working with multiple AI services while providing extensibility through plugins. This allows developers to…
1 min read
Prompt Engineering Patterns for Enterprise RAG Systems
Poorly designed prompts lead to hallucinations, ignored context, or responses that fail to meet business requirements. A systematic approach to prompt…
1 min read
Batch Processing AI Workloads with Azure Batch and OpenAI
Track job progress and handle failures with proper retry logic for robust batch AI processing at scale.
1 min read
GraphRAG: Combining Knowledge Graphs with Retrieval-Augmented Generation
GraphRAG excels when your questions involve multi-hop reasoning, relationship discovery, or when entity connections matter more than document similarity.
1 min read
Building Copilot Extensions with Azure OpenAI Assistants API
The Assistants API provides the foundation for building powerful, context-aware AI assistants that integrate with enterprise systems.
1 min read
Cost Optimization Strategies for Azure OpenAI Deployments
Balance cost and quality by routing requests to appropriate models based on task requirements.
1 min read
Building AI Agents with AutoGen and Azure OpenAI
AutoGen enables building sophisticated AI systems that can decompose complex tasks and collaborate to find solutions.
1 min read
Azure Functions with AI: Building Serverless AI Applications
Configure appropriate timeout settings for AI operations and implement retry logic for transient failures. Consider using premium plans for…
1 min read
Implementing RAG with LangChain and Azure OpenAI
Experiment with chunk sizes, overlap, and retrieval strategies. Hybrid search combining semantic and keyword matching often provides the best results for…
1 min read
Semantic Kernel: Building AI Orchestration with .NET
Semantic Kernel provides a clean abstraction for building AI-powered features while maintaining flexibility to switch between different LLM providers.
1 min read
Fine-Tuning Azure OpenAI Models: When and How to Customize
Fine-tuning is appropriate when you need consistent formatting, domain-specific language understanding, or reduced prompt lengths. However, it requires…
1 min read
Prompt Engineering Patterns for Enterprise Applications
Include examples in prompts to guide model behavior for domain-specific tasks. This is especially effective for classification and formatting tasks where…
1 min read
Building Multi-Modal AI Applications with GPT-4 Vision and Audio
Multi-modal requests consume more tokens and have higher latency. Use the detail parameter wisely: "low" for quick analysis, "high" for detailed extraction…
1 min read
AI Safety Guardrails: Implementing Content Filtering in Azure OpenAI
Maintain comprehensive logs of all content filtering decisions for compliance and incident investigation. Every blocked request should be logged with…
1 min read
Fine-Tuning GPT Models: When and How to Customize
Fine-tuning makes sense when you need consistent formatting, domain-specific terminology, or significant behavior changes that prompting cannot achieve…
1 min read
Structured Output with Azure OpenAI: JSON Mode and Response Formats
JSON mode guarantees the model outputs valid JSON, though you still need to specify the schema in your prompt.
1 min read
Function Calling Patterns in Azure OpenAI
Functions are defined as JSON schemas that describe parameters and their types. Well-designed function schemas and clear descriptions ensure the model calls…
1 min read
GPT-4o Vision: Building Image Analysis Applications
GPT-4o vision provides remarkable understanding of images, but always validate outputs for critical applications. Combine with traditional computer vision…
1 min read
Cost Optimization Strategies for Azure OpenAI Workloads
Semantic caching avoids redundant API calls for similar queries. Reduce prompt length by removing unnecessary context. Use batch processing with the Batch…
1 min read
Azure OpenAI o1 Models: Reasoning at Scale for Complex Problems
The o1 models are designed for tasks requiring deep reasoning: mathematical proofs, code debugging, scientific analysis, and strategic planning. They take…
1 min read
Building RAG Applications with Azure AI Search and GPT-4o
RAG works by first retrieving relevant documents from a search index, then passing those documents as context to an LLM for generation. This approach…
1 min read
Building Knowledge Graphs with Azure OpenAI and Neo4j
Graph RAG significantly improves answers for questions involving relationships, hierarchies, and multi-hop reasoning.
1 min read
Fine-Tuning GPT-4o for Domain-Specific Tasks on Azure
After training completes, evaluate on a held-out test set before deploying. Monitor the fine-tuned model's performance against the base model to ensure…
1 min read
Prompt Caching with Claude and Azure OpenAI: Reducing Latency and Costs
Track cache hit rates in your monitoring. Applications with high context reuse typically see 60-80% cache hit rates, dramatically reducing per-request costs.
1 min read
GPT-4o Vision for Document Intelligence: Building Smart OCR Pipelines
For multi-page documents, process pages in parallel and use a synthesis step to merge extracted data. GPT-4o handles cross-page references like "continued…
1 min read
Azure AI Updates: What's New in February 2025
Stay current with Azure AI updates to leverage the latest capabilities for your data and AI applications.
8 min read
LLMs on an Event Hubs Stream: Tiered Triage with Azure Functions
How to put Azure OpenAI inside an Event Hubs pipeline without stalling partitions: batch triggers, a cheap triage tier, structured outputs, fail-closed errors.
7 min read
Turning Call Recordings into Data with Azure AI Speech and GPT-4o
A batch pipeline for call and meeting audio: Azure AI Speech transcription with diarization, then GPT-4o structured outputs into rows you can query.
6 min read
Reading Charts and Dashboards with GPT-4o: What to Trust
Where GPT-4o vision helps data teams read charts and dashboard screenshots, what it gets wrong, what each image costs, and how to check its numbers in code.
7 min read
Multimodal RAG on Azure: Caption PDF Figures Before You Embed Them
Make diagrams and charts in PDFs retrievable with Document Intelligence 4.0 figure extraction, GPT-4o descriptions and Azure AI Search hybrid queries.
9 min read
Azure OpenAI in Production, January 2025: Let the Platform Do the Work
Deployment types, keyless auth, SDK retries, APIM token limits and Global Batch: what to build on Azure OpenAI in January 2025, and what to stop building.
7 min read
LLM-Drafted Data Dictionaries: Profile First, Describe Second
Draft column descriptions for an Azure SQL database with Azure OpenAI, grounded in profiling stats, reviewed by a data owner, then stored as MS_Description.
8 min read
LLM-Drafted Tests for PySpark: The Model Writes, pytest Judges
Use Azure OpenAI structured outputs to draft edge-case test data for PySpark transforms, review it like code, and let pytest and assertDataFrameEqual decide.
7 min read
LLM-Drafted Data Quality Rules: Let the Model Propose, Not Enforce
Use Azure OpenAI structured outputs to draft Great Expectations rules from a data profile, then backtest and review them before they ever gate a pipeline.
7 min read
Generating dbt Models with Azure OpenAI: Let dbt Be the Judge
Use Azure OpenAI structured outputs to draft dbt models and YAML, then gate every draft with dbt build so nothing untested reaches review.
6 min read
Drafting a Star Schema with GPT-4o and Structured Outputs
Use GPT-4o structured outputs to draft a star schema from requirements, check it with plain code, and generate Fabric Warehouse DDL that respects its limits.
7 min read
Natural Language over a Power BI Model: Let the LLM Pick, Not Write
Answer plain-English questions from a Power BI semantic model by having Azure OpenAI choose measures and filters, then run them with semantic link.
7 min read
GraphRAG 1.x: When Global Search Justifies the Indexing Bill
How Microsoft's GraphRAG 1.x builds a knowledge graph, what local, global and DRIFT search do, and when its indexing cost is worth paying over vector RAG.
8 min read
RAG Beyond Top-K: Retrieval Upgrades Worth Making in January 2025
Which RAG retrieval upgrades Azure AI Search gives you as GA or preview in January 2025, which you build yourself, and the order I'd add them in.
7 min read
Prompting in 2025: What Changes with o1 and Structured Outputs
Which prompt techniques still earn their place on Azure OpenAI in January 2025, which moved into the API, and which now work against reasoning models like o1.
8 min read
Five LLM Application Patterns, From Least to Most Autonomy
A practical ladder of LLM patterns on Azure in January 2025: structured outputs, grounded RAG, tool workflows, self-review and agents.
7 min read
o1 Is GA, o3 Is Announced: Reasoning Models in January 2025
What changed from o1-preview to o1 (2024-12-17), what o3 actually is today, and how I'd use reasoning_effort and developer messages right now.
7 min read
Gemini 2.0 Flash Experimental: What It Means for Azure-First Teams
What Google actually shipped with Gemini 2.0 Flash in December 2024, what is still experimental, and when an Azure-first team should run a trial.
7 min read
GPT-5 in January 2025: Reading the Signals Instead of Guessing
What o1, the o3 announcement and reports on Orion actually tell us about GPT-5 in January 2025, and what Azure OpenAI teams should build while they wait.
8 min read
Agentic Workflows That Survive Restarts with Durable Functions
Run an LLM tool-calling loop inside Azure Durable Functions so it checkpoints each step, stays within a step budget and waits hours for human approval.
1 min read
Azure OpenAI November 2024 Updates: What's New for Enterprise AI
Batch processing can reduce costs by up to 50% for workloads that don't need immediate responses.
1 min read
Tool Use Patterns for AI Agents
Tools extend what AI agents can do beyond text generation. Today I'm exploring patterns for effective tool use in production agents.
3 min read
GPT-4o: Multimodal AI Gets Real-Time
Total latency: 2-5 seconds. GPT-4o processes audio natively - 232ms average response time. That's human conversational speed. The model understands tone…
1 min read
Deploying GPT-4o on Azure OpenAI Service
1. Navigate to your Azure OpenAI resource 2. Go to Model deployments Deploy model 3. Select from the model list 4. Configure deployment settings 5. Deploy…
1 min read
The Future of Multimodal AI: What's Next After GPT-4 Vision
Each modality is handled separately, then combined. This works, but has latency and integration challenges.
1 min read
DALL-E 3 Patterns: Effective Image Generation for Enterprise
I experimented with DALL‑E 3 for enterprise image generation; these patterns capture what scaled well and what to avoid.
1 min read
GPT-4 Vision Patterns: Building Visual AI Applications
I've used GPT-4 Vision in real projects; these are the practical patterns that helped move visual AI from experiment to production.
1 min read
Provisioned Throughput Deep Dive: Architecture and Implementation
1. Multi region deployment Distribute for resilience 2. Monitor continuously Track utilization and latency 3. Plan scaling windows PTU changes aren't…
1 min read
Azure OpenAI PTU: Provisioned Throughput Units Explained
Aspect Pay As You Go PTU Pricing Per token Per PTU hour Throughput Shared, variable Dedicated, consistent Latency Variable Lower, consistent Commitment None…
1 min read
Prompt Caching: Reducing Latency and Cost for Repetitive Contexts
Many applications send the same system prompt and context repeatedly: 1. Identify common prefixes System prompts, few shot examples 2. Batch similar…
1 min read
Token Cost Analysis: Understanding and Managing LLM Expenses
Token cost analysis enables informed decisions about LLM usage. Track, analyze, and optimize to keep costs predictable and manageable.
1 min read
Cost Optimization for LLMs: Strategies That Work
1. Cache aggressively Identical queries don't need re processing 2. Tier your models Match model to task complexity 3. Optimize prompts Shorter prompts =…
1 min read
Multi-LLM Strategies: Building Resilient AI Applications
Resilience : Failover when one provider has outages Cost optimization : Route to cheaper models when possible Capability matching : Use best model for each…
1 min read
Gemini vs GPT: Practical Comparison for Enterprise Applications
I spent time comparing Gemini and GPT in enterprise settings; below are the practical differences that should influence model choice.
7 min read
Integrated Vectorization in Azure AI Search: What It Does and Doesn't Do
What integrated vectorization in Azure AI Search actually removes from a RAG pipeline, what it leaves behind, and when I'd still embed in my own code.
6 min read
CrewAI 0.1 on Azure OpenAI: Roles, Tasks and Current Limits
A practical look at CrewAI 0.1.x with Azure OpenAI: how agents, tasks and sequential crews work, what's missing today, and when to choose it.
7 min read
AutoGen 0.2 on Azure OpenAI: Termination, Code Execution and Tools
A practical look at AutoGen 0.2 with Azure OpenAI: wiring the config, stopping runaway conversations, sandboxing generated code and registering tools.
7 min read
When to Split One AI Agent into Several: A Supervisor Pattern
When a single GPT-4 agent stops being reliable, split it into specialists. A framework-free supervisor pattern on Azure OpenAI, with trade-offs and limits.
6 min read
Self-Correcting RAG: Grade Retrieval Before You Answer
How to add retrieval grading, query rewriting, and groundedness checks to RAG on Azure OpenAI and Azure AI Search without tripling latency or cost.
7 min read
Owning the Loop: AI Assistant Architecture on Chat Completions
How to structure a production AI assistant on Azure OpenAI Chat Completions: conversation state, retrieval, tool calling and the orchestration loop you own.
7 min read
Shipping on GPT-4 Turbo Preview in Azure OpenAI: A Production Checklist
GPT-4 Turbo is still a preview model on Azure OpenAI; here's how I'd deploy, budget, retry and test around 1106-preview before real users touch it.
7 min read
Enterprise AI in 2024: Five Bets Against What Is GA on Azure
Five enterprise AI bets for 2024, each checked against what is generally available or still in preview on Azure as of 1 January 2024.
1 min read
Rate Limit Management for AI APIs
Tomorrow we'll explore token estimation techniques. Rate Limiting Patterns Circuit Breaker Pattern Azure OpenAI Best Practices
1 min read
Azure OpenAI Quotas: Understanding and Managing Limits
Tomorrow we'll explore rate limit management strategies. Azure OpenAI Quotas Quota Management Rate Limit Best Practices
1 min read
Data Analysis with AI: Practical Patterns and Techniques
The Code Interpreter release this month changed how I approach quick data analysis tasks. Not the deep, production-grade EDA that belongs in a Fabric…
1 min read
Azure OpenAI Code Interpreter: AI-Powered Data Analysis
Tomorrow we'll explore ChatGPT Code Interpreter capabilities. Azure OpenAI Assistants Code Interpreter Guide Azure OpenAI Pricing
1 min read
Circuit Breakers for AI Applications: Preventing Cascade Failures
This concludes our June 2023 series on Azure, Data, and AI topics. We covered: Microsoft Fabric deep dives (Lakehouse, Delta Lake, OneLake) Power BI Direct…
1 min read
Fallback Patterns for AI Applications
Fallback patterns ensure continuous service availability. Tomorrow, I will cover circuit breakers for AI applications.
1 min read
Retry Strategies for AI Applications
Azure OpenAI rate limits in mid-2023 are set per deployment per region, and hitting them is a normal operating condition rather than an exceptional one for…
1 min read
Error Handling for AI Applications: Building Resilient Systems
Robust error handling ensures AI applications remain useful even during failures. Tomorrow, I will cover retry strategies in more depth.
1 min read
Response Streaming: Real-Time LLM Output
Streaming improves the user experience significantly. Tomorrow, I will cover error handling for AI applications.
1 min read
Caching Strategies for LLM Applications
Effective caching significantly reduces LLM costs and latency. Tomorrow, I will cover response streaming patterns.
1 min read
Token Budgeting: Cost-Effective LLM Applications
Token costs for GPT-4 in mid-2023 are real enough to design around: roughly $0.03 per 1K input tokens and $0.06 per 1K output tokens for the 8K context…
1 min read
Context Pruning: Managing Token Limits in LLM Applications
Effective context pruning ensures your LLM applications work reliably. Tomorrow, I will cover token budgeting strategies.
1 min read
Conversation Management: Building Production Chat Systems
Robust conversation management enables reliable production chat systems. Tomorrow, I will cover context pruning strategies.
1 min read
Multi-Turn Conversations: Managing State in AI Applications
Multi-turn conversation management is essential for production chatbots. Tomorrow, I will cover conversation management strategies.
1 min read
Tool Use Patterns for AI Agents: Effective Function Design
The most consequential design decision in a function-calling agent isn't the model you choose — it's how you write the tool definitions. The model selects…
1 min read
Building AI Agents: From Chatbots to Autonomous Assistants
Function calling, released last week with the 0613 model versions, gives AI agents a proper foundation — and I've been rebuilding some agent prototypes to…
1 min read
Azure OpenAI Function Calling Patterns: Building Intelligent Agents
Function calling patterns enable sophisticated AI agents. Tomorrow, I will cover building AI agents in more depth.
1 min read
Azure AI Studio Preview: The Future of AI Application Development
Azure AI Studio provides a comprehensive environment for building, testing, and deploying AI applications. Tomorrow, I will cover Prompt Flow in more detail.
1 min read
Mastering Function Calling in Azure OpenAI: A Deep Dive
Function calling transforms GPT models into powerful agents that can interact with the real world. Tomorrow, I will cover Azure AI Studio in more detail.
1 min read
Azure OpenAI Updates at Build 2023: What Developers Need to Know
Azure OpenAI is becoming the enterprise-grade platform for building generative AI applications. Tomorrow, I will cover building plugins for ChatGPT.
7 min read
RAG on Azure OpenAI: Grounding GPT-3 Answers in Your Own Documents
What retrieval augmented generation is, why it beats fine-tuning for company knowledge, and a minimal loop on Azure OpenAI completions in January 2023.
7 min read
Azure Cognitive Search and Embeddings Before Native Vector Search
Azure Cognitive Search can't search vectors in January 2023. Here's how to re-rank its keyword results with Azure OpenAI embeddings, and when not to bother.
6 min read
Qdrant 0.11 with Azure OpenAI: Point IDs, Payload Indexes and Filters
Run Qdrant 0.11 with Azure OpenAI embeddings: valid point IDs, payload indexes, filtered HNSW search, memory settings, and when Qdrant isn't the right choice.
7 min read
Milvus 2.2 with Azure OpenAI Embeddings: A Self-Hosted Start
Run Milvus 2.2 standalone, load Azure OpenAI embeddings with pymilvus, and learn the schema, index, consistency and delete rules before you self-host it.
7 min read
Weaviate 1.17 with Azure OpenAI: Bring Your Own Vectors
Run Weaviate 1.17 with embeddings from Azure OpenAI: schema choices, batch imports, filtered vector search, BM25 and the hybrid search caveats to know first.
8 min read
Pinecone With Azure OpenAI Embeddings: Design the Index First
Set up a Pinecone index for Azure OpenAI embeddings: dimensions, pod types, namespaces, metadata indexing and idempotent upserts, plus when to skip it.
8 min read
Semantic Search Prototype: Azure OpenAI Embeddings and NumPy
Build a small semantic search prototype on Azure OpenAI embeddings and NumPy, measure it with hit rate@k, add BM25, and know when to move to a real index.
8 min read
Text Embeddings on Azure OpenAI: What They Are and When to Use Them
What text embeddings are, which Azure OpenAI embeddings models you can use in January 2023, how to compare vectors, and when embeddings are the wrong tool.
7 min read
Streaming Azure OpenAI Completions from the API to the Browser
How stream=True works on the Azure OpenAI 2022-12-01 Completions API, what you give up by streaming, and how to relay tokens to a browser without losing any.
7 min read
Calling Azure OpenAI over Plain HTTP: The 2022-12-01 REST Contract
What the GA 2022-12-01 Azure OpenAI REST API looks like on the wire: URLs, auth headers, completions, embeddings, errors and when to skip the SDK.
7 min read
Azure OpenAI Is GA: What Changed on 16 January and What Didn't
Azure OpenAI Service reached GA on 16 January 2023: what that changes for production plans, what still needs an approved application, and how to proceed.
7 min read
Calling Azure OpenAI from C# Before There's an Official .NET SDK
Azure OpenAI is GA but has no official .NET SDK yet. How I'd build a thin, typed C# client over the 2022-12-01 REST API that is easy to delete later.
7 min read
Azure OpenAI from Python: Setting Up the openai 0.26 Library Properly
How to configure the openai 0.26 Python library for Azure OpenAI in January 2023: API keys or Azure AD tokens, deployment names, timeouts, retries and async.
6 min read
Completions Now, Chat Later: Keeping Azure OpenAI Code Portable
Azure OpenAI is GA with a Completions API only, and ChatGPT is coming soon. How to decide what needs chat and keep your prompt code ready for the change.
6 min read
Temperature and Top-P in Azure OpenAI: What They Actually Change
How temperature and top_p reshape token sampling in Azure OpenAI completions, why to tune one not both, and how to measure the effect before you ship.
8 min read
No System Prompt Yet: Instruction Preambles on Azure OpenAI
Azure OpenAI completion models have no system role yet. How to design, budget, version and test an instruction preamble that does that job instead.
8 min read
Chain-of-Thought Prompting on Azure OpenAI: Gains, Costs and Limits
What the 2022 chain-of-thought papers show, how to apply zero-shot CoT and self-consistency on Azure OpenAI completions, and when the extra tokens don't pay.
8 min read
Few-Shot Prompts on Azure OpenAI: Choosing and Ordering Examples
How to pick, order and budget few-shot examples for Azure OpenAI completion models, with the biases to watch for and when to stop adding examples.
7 min read
Prompts Are Production Code: Four Rules for Azure OpenAI Completions
Four prompt rules that hold up against real user input on Azure OpenAI completions: positive instructions, strict formats, delimiters and roles, plus tests.
8 min read
Counting and Capping Tokens in the Azure OpenAI Preview
How to count tokens with tiktoken, size max_tokens, log real usage and cap spend for Azure OpenAI completion models in the January 2023 preview.
7 min read
Throttling in the Azure OpenAI Preview: Pacing Per-Deployment Requests
Azure OpenAI preview limits are requests per second per deployment. How to pace calls, retry 429s properly and scale out with the openai 0.26 Python library.
7 min read
Handling Content Filtering in the Azure OpenAI Completions API
How Azure OpenAI's preview content filter shows up in code: HTTP 400 prompts, content_filter finish reasons, and why you still need your own guardrails.
8 min read
Responsible AI on Azure OpenAI: What the Platform Covers and What You Own
Azure OpenAI's preview ships with use-case review, content filtering and abuse monitoring. Here is where those controls stop and your application's begin.
7 min read
Azure OpenAI vs OpenAI API: What Differs in January 2023
Azure OpenAI or OpenAI's own API in January 2023? Access, models, data use, cost and code compared, plus one Python client that can target either.
8 min read
Locking Down an Azure OpenAI Resource: Network, Identity, Keys, Logs
A security baseline for an Azure OpenAI preview resource in January 2023: private endpoints, Azure AD auth, customer-managed keys, logging and policy.
7 min read
ChatGPT-Style Chat on Azure OpenAI Without a Chat API
There is no ChatGPT API yet. Here is how to build multi-turn chat on Azure OpenAI's Completions API: transcripts, state, token budgets and streaming.
8 min read
GPT-3.5 on Azure OpenAI: What You Get Before text-davinci-003
What GPT-3.5 means on Azure OpenAI in January 2023: which models are in the catalogue, what they cost, and how to design for text-davinci-003's arrival.
8 min read
Azure OpenAI Service in Preview: What to Sort Out Before GA
Azure OpenAI is still a limited-access preview as 2023 starts. Here is what to settle now on access, identity, data handling and code so you're ready for GA.