8 min read
GPT-6 Landed in Foundry: Upgrade Discipline Beats Upgrade Speed
GPT-6 arrived in Foundry ten weeks after GPT-5.6. Pin versions, gate on real queries, shadow traffic and plan retirements before you switch tiers on day one.
LLM
157 articles
9 min read
GPT-5.6 Sol, Terra or Luna: Pick the Tier per Job, Not per App
GPT-5.6 in Foundry has a 5x price spread across Sol, Terra and Luna. Choose a tier for each workflow step, back it with eval gates, then pick deployment types.
8 min read
Why an LLM Response Cache Moves Your p50 but Rarely Your p95
A response cache only speeds up the requests it hits, so the tail stays slow until hit rates get very high. Here's the arithmetic and what fixes p95 instead.
7 min read
Does Your LLM Eval Score Predict Anything the Business Cares About?
Before an eval score becomes a product metric, join its verdicts to real outcomes like escalations and re-asks, and check whether failing answers do worse.
8 min read
Splitting an AI Journey's Latency Budget Across Its Steps
A journey-level latency budget only works when each step gets a cap, a timeout and a fallback; here is how to split one across a RAG request.
8 min read
Turning Production Queries into an LLM Eval Set You Can Trust
How I sample real user queries by stratum, redact PII before anyone reads them, split off a holdout and keep the eval set current as the product changes.
8 min read
Four LLM Caches and How to Tell Which One Will Pay Off
Prompt, response, semantic and retrieval caches save different money and add different risks. Here's a break-even check to run before you build one.
7 min read
Raising Groundedness Without Guesswork: Label the Failures First
Once retrieval is ruled out, ungrounded answers fall into a few repeatable patterns: label them, fix one at a time and track completeness too.
7 min read
Latency Budgets as a Cost Control: Pricing Each AI User Journey
Set a latency budget per user journey, then let it choose the Azure OpenAI deployment tier, model and reasoning effort you actually pay for.
8 min read
Designing LLM Test Cases That Catch Hallucinations
Most eval sets only ask questions the documents can answer. Add unanswerable, false-premise and stale-context cases, each with an expected behaviour.
8 min read
Where LLM Latency Goes, and Which Token Cuts Actually Help
Input and output tokens cost and delay answers differently, so split latency into first-token and generation time before trimming anything.
8 min read
Quality SLOs for LLM Features: An Error Budget for Bad Answers
Treat answer quality like availability: define a good answer, agree a target with the product owner, and let the error budget decide when changes stop.
7 min read
A Token Waste Audit: Cut What the Answer Doesn't Use
Profile each LLM request by segment, rank the cuts by quality risk, and keep only the token savings that survive a paired evaluation against the baseline.
8 min read
Gating LLM Changes on Groundedness Flips, Not Fluency Averages
How I compare a baseline and a candidate prompt or model on frozen context, count groundedness regressions per row, and treat fluency as a tie-breaker.
8 min read
LLM-as-a-Judge: Check the Judge Before You Trust the Score
An LLM judge has its own biases: how to write pass/fail criteria, measure its kappa and false-pass rate against human labels, and know when to skip it.
8 min read
Learning AI Without a PhD: What Transfers From Cloud Engineering
How I moved from cloud and data engineering into AI work without a maths degree: what I learnt in which order, what I skipped, and where engineers go wrong.
8 min read
AI Security Basics: Keys, Quotas and Logs Before Prompt Tricks
The AI security failures that actually cost money are leaked keys, unbounded token spend and sensitive data in logs. Fix those before prompt defences.
8 min read
Testing AI Systems: Which Tests Run Where, and Why
A layered test strategy for LLM apps: stubbed unit tests per commit, pass-rate gates on PRs, nightly evaluator runs and red teaming before release.
8 min read
Cost Per Query: Measure the Unit Cost Before You Cut It
Treat cost per query as the unit metric for an LLM app: decompose it from the usage object, then cut it in order with model choice, caching, context and batch.
8 min read
LLM Observability: The Few Signals That Actually Matter
Which LLM signals deserve a dashboard or a page: tokens by feature, latency split by phase, throttling, and sampled quality, using OpenTelemetry on Azure.
7 min read
Eight Prompt Engineering Myths to Drop in 2026
Eight prompt engineering myths worth dropping in 2026, from verbose prompts to reflexive chain-of-thought and temperature rules, and what to measure instead.
1 min read
2025 AI Year in Review: The Year Agents Became Real
The most significant shift this year was the maturation of autonomous AI agents. What started as experimental frameworks evolved into production-ready…
1 min read
AI-Powered Code Review: Integrating LLMs into Development Workflows
While linters catch syntax issues, AI reviewers understand context, identify logical errors, suggest architectural improvements, and explain complex code…
1 min read
Semantic Caching for LLM Applications: Reducing Costs and Latency
Traditional caching requires exact matches. Users asking "What is Azure?" and "Can you explain Azure?" would generate two separate API calls. Semantic…
1 min read
Monitoring AI Applications: Observability Patterns for LLM Systems
LLM systems have unique characteristics: non-deterministic outputs, variable latency, complex cost models, and quality metrics that require semantic…
1 min read
Model Fine-Tuning on Azure: When and How to Customize LLMs
Fine-tuning makes sense when you need consistent style, specialized terminology, or improved performance on specific tasks that prompt engineering cannot…
1 min read
Testing LLM Applications: Strategies Beyond Traditional Unit Tests
LLM outputs vary between runs and model versions. Testing must focus on behavioral properties rather than exact string matching, while still catching…
1 min read
Testing AI Applications: Strategies for LLM-Based Systems
Test AI components within the full application context, including error handling, timeout behavior, and integration with downstream systems.
1 min read
Fine-Tuning Azure OpenAI Models: When and How to Customize
Fine-tuning is appropriate when you need consistent formatting, domain-specific language understanding, or reduced prompt lengths. However, it requires…
1 min read
Prompt Engineering Patterns for Enterprise Applications
Include examples in prompts to guide model behavior for domain-specific tasks. This is especially effective for classification and formatting tasks where…
1 min read
Retrieval-Augmented Generation: Chunking Strategies for Better Results
The best chunking strategy depends on your document types. Technical documentation benefits from header-aware chunking. Conversational content works well…
1 min read
Prompt Injection Defense: Securing LLM Applications
Prompt injection occurs when user input is interpreted as instructions rather than data. Attackers can attempt to override system prompts, extract…
1 min read
Responsible AI: Content Safety Filters for LLM Applications
Content safety is non-negotiable for production AI applications. Implement multiple layers of filtering for both inputs and outputs to protect users and…
1 min read
AI Agent Memory: Implementing Long-Term Context
Long-term memory transforms one-shot interactions into ongoing relationships. Users feel understood when the AI remembers their preferences, past issues…
1 min read
Prompt Engineering: System Prompts That Actually Work
Invest time in crafting and testing your system prompts. They are the foundation of consistent, high-quality AI interactions.
1 min read
Semantic Caching for LLM Applications: Reducing Costs and Latency
Unlike exact-match caching, semantic caching uses embeddings to find similar queries even when worded differently.
1 min read
Fine-Tuning with LoRA: Efficient Model Customization
Instead of updating all model weights, LoRA injects trainable low-rank matrices into transformer layers. These adapters capture task-specific knowledge…
1 min read
Multi-Agent Systems: Orchestrating Specialized AI Agents
Multi-agent architectures shine for tasks requiring diverse expertise: research reports, code reviews, complex analysis, and creative projects. The key is…
1 min read
Prompt Engineering: Structured Output with JSON Mode
For even stricter control, use the structured outputs feature with a JSON schema. JSON mode transforms LLMs from conversational tools into reliable data…
1 min read
Fine-Tuning GPT Models: When and How to Customize
Fine-tuning makes sense when you need consistent formatting, domain-specific terminology, or significant behavior changes that prompting cannot achieve…
1 min read
Building AI Agents with Tool Use: A Practical Architecture
At its core, an AI agent follows a simple loop: observe, think, act, repeat. Tools should be focused, well-documented, and handle errors gracefully.
1 min read
Prompt Engineering: Chain-of-Thought Techniques for Complex Reasoning
When faced with multi-step problems, LLMs perform better when prompted to reason step-by-step rather than jump directly to answers.
1 min read
Data Quality for AI: Ensuring Clean Data for ML and LLM Applications
Quality encompasses completeness, accuracy, consistency, timeliness, and validity. Each dimension requires specific checks.
1 min read
Prompt Engineering Best Practices for Enterprise Applications
A well-structured prompt includes context, instructions, examples, and output format specifications.
1 min read
Azure OpenAI o1 Models: Reasoning at Scale for Complex Problems
The o1 models are designed for tasks requiring deep reasoning: mathematical proofs, code debugging, scientific analysis, and strategic planning. They take…
1 min read
Getting Started with Semantic Kernel 2.x: A Practical Introduction
Semantic Kernel is an SDK that integrates Large Language Models (LLMs) with conventional programming languages. Version 2.x introduces a more streamlined…
1 min read
Observability for LLM Applications: Tracing, Metrics, and Debugging
Log prompts and responses for failed interactions to a secure store for debugging. Implement sampling for successful calls to control storage costs while…
1 min read
Structured Output with JSON Mode: Reliable Data Extraction from LLMs
Structured outputs eliminate parsing errors. Add business validation for semantic correctness - the schema ensures format, you ensure meaning.
1 min read
Implementing Semantic Caching for LLM Applications
Use Redis Vector Search for efficient similarity matching at scale. Tune the similarity threshold based on your use case - higher values ensure more precise…
1 min read
Streaming LLM Responses: Patterns for Real-Time User Experience
Add a blinking cursor effect while streaming to indicate ongoing generation. Users perceive streaming responses as 3-5x faster than equivalent non-streaming…
1 min read
Speculative Decoding: Accelerating LLM Generation
Speculative decoding can achieve 2-3x speedup without any quality degradation.
1 min read
SLM vs LLM Strategies: Choosing the Right Model Size
The right model size depends on your specific requirements, not just capabilities.
1 min read
LLM Fine-Tuning on Azure: When and How to Customize Models
Fine-tuning is a powerful tool when used appropriately, but prompt engineering often achieves similar results faster.
1 min read
Context Window Management: Maximizing LLM Effectiveness
Strategic context management enables handling of complex, long-context scenarios.
1 min read
SLM vs LLM: Choosing the Right Model Size Strategy
The right model isn't always the biggest one. Match model capability to task requirements for optimal cost, latency, and quality.
7 min read
Generating dbt Models with Azure OpenAI: Let dbt Be the Judge
Use Azure OpenAI structured outputs to draft dbt models and YAML, then gate every draft with dbt build so nothing untested reaches review.
8 min read
RAG Beyond Top-K: Retrieval Upgrades Worth Making in January 2025
Which RAG retrieval upgrades Azure AI Search gives you as GA or preview in January 2025, which you build yourself, and the order I'd add them in.
7 min read
Prompting in 2025: What Changes with o1 and Structured Outputs
Which prompt techniques still earn their place on Azure OpenAI in January 2025, which moved into the API, and which now work against reasoning models like o1.
7 min read
o1 Is GA, o3 Is Announced: Reasoning Models in January 2025
What changed from o1-preview to o1 (2024-12-17), what o3 actually is today, and how I'd use reasoning_effort and developer messages right now.
7 min read
Gemini 2.0 Flash Experimental: What It Means for Azure-First Teams
What Google actually shipped with Gemini 2.0 Flash in December 2024, what is still experimental, and when an Azure-first team should run a trial.
7 min read
Claude 4 Speculation: What to Plan For While Anthropic Is Quiet
No Claude 4 is announced yet: what Anthropic actually shipped in late 2024, what it hints at, and how to stay ready without guessing.
7 min read
GPT-5 in January 2025: Reading the Signals Instead of Guessing
What o1, the o3 announcement and reports on Orion actually tell us about GPT-5 in January 2025, and what Azure OpenAI teams should build while they wait.
1 min read
LLMOps Emergence: Operating Large Language Models in Production
LLMOps is essential for reliable LLM applications. Start with prompt management and evaluation, then add observability and cost tracking as you scale.
1 min read
Azure AI Model Catalog: Exploring the Latest Model Options
The Model Catalog gives you flexibility to choose the right model for each use case while maintaining a consistent API. Experiment with different models to…
2 min read
OpenAI o1 Reasoning Models: A New Paradigm for Complex Problem Solving
The reasoning tokens are where the model works through the problem step-by-step, similar to how humans solve complex problems.
1 min read
Building Custom LLM Observability: DIY Solutions
Building custom observability gives you full control over your data and features. Start simple and add complexity as your needs grow.
1 min read
Arize AI for Production LLM Monitoring
Arize provides comprehensive production monitoring for LLMs with enterprise features like drift detection, alerting, and detailed analytics. It's ideal for…
1 min read
Phoenix Observability: Local-First LLM Tracing
Phoenix provides powerful local-first observability that keeps your data private while offering the visualization and analysis capabilities needed for…
1 min read
MLflow Tracing for LLM Applications: Open Source Observability
MLflow provides a solid open-source foundation for LLM observability. Its strength lies in the familiar MLOps workflow and integration with the broader ML…
1 min read
Weights and Biases for LLM Applications: Complete Guide
Weights & Biases provides a comprehensive platform for LLM observability, evaluation, and collaboration. Its strength lies in combining experiment tracking…
1 min read
The Evolution of Tool Use in AI: From Functions to Computer Control
The evolution of tool use in AI represents a fundamental shift from constrained function execution to general-purpose computer interaction. This trajectory…
1 min read
Agentic AI Capabilities: From Chatbots to Autonomous Agents
Agentic capabilities transform AI from a question-answering system into an autonomous problem-solver. The key is combining planning, tools, memory, and…
1 min read
Error Handling in LLM Applications: A Comprehensive Guide
Robust error handling is what separates prototypes from production systems. Invest in comprehensive error handling early to avoid painful debugging later.
2 min read
Leveraging Azure OpenAI with Claude Dev and Continue in VSCode
As a developer constantly evaluating new tools to enhance productivity, I've recently implemented a setup that combines Azure OpenAI with two VSCode…
1 min read
Reliable Data Extraction with LLMs: Patterns and Practices
Reliable extraction requires careful schema design, multi-pass validation, and explicit handling of uncertainty. These patterns help you build extraction…
1 min read
Thinking Tokens: Understanding o1's Hidden Reasoning Process
Understanding thinking tokens helps you make informed decisions about when o1's extended reasoning is worth the investment.
1 min read
Understanding Reasoning in AI: From Pattern Matching to Problem Solving
Traditional LLMs like GPT-4o are sophisticated pattern matchers. They predict the next token based on learned patterns from training data. While incredibly…
2 min read
The Future of Reasoning in AI: What's Coming Next
This could dramatically improve performance on complex tasks. The next breakthrough in AI capabilities is likely to come from better reasoning, not just…
2 min read
Short-Term Agent Memory: Context Window Management
Sounds like a lot, but it fills up quickly with conversation history, system prompts, tool outputs, and retrieved documents.
1 min read
LangGraph Introduction: Building Stateful AI Agents
LangGraph provides these capabilities through a graph-based execution model.
1 min read
LangChain Updates: What's New in July 2024
LangChain 0.2.x Stability LCEL (LangChain Expression Language) Maturity Azure AI Search Vector Store
1 min read
Semantic Router: Intent-Based LLM Routing
Semantic routing compares the meaning of a user's query against a set of example utterances. When the query is semantically similar to examples for a…
1 min read
Latency-Based LLM Routing: Real-Time AI Applications
Latency has multiple components: 1. Network latency : Request travel time 2. Queue time : Waiting for processing 3. Time to first token (TTFT) : Initial…
4 min read
Quality-Based LLM Routing: Maximizing Output Quality
Quality isn't one dimensional. Consider: Accuracy : Factual correctness Completeness : Covering all aspects Coherence : Logical flow and consistency…
1 min read
Cost-Based LLM Routing: Optimizing AI Spend
The difference is dramatic. A 1000-token task costs $0.02 with GPT-4o but $0.0002 with GPT-4o-mini.
5 min read
LLM Routing Strategies for Production Systems
The simplest approach: fixed rules based on task type. Pros: Simple, predictable, easy to debug Cons: Doesn't adapt, requires manual tuning
1 min read
Multi-Model Orchestration: Using Multiple LLMs Effectively
Different tasks need different capability levels. Using GPT-4o for simple classifications wastes money.
2 min read
Claude 3.5 Sonnet: Anthropic's New Benchmark in AI
Performance That Competes Claude 3.5 Sonnet outperforms Claude 3 Opus on most benchmarks while being significantly faster and cheaper. It's positioned as a…
1 min read
Foundation Model APIs in Databricks: Enterprise LLM Access
Databricks Foundation Model APIs provide enterprise-ready access to state-of-the-art LLMs. This guide covers using these APIs for building AI applications.
1 min read
The ai_query Function: Custom LLM Calls in Databricks SQL
The aiquery() function enables custom LLM interactions directly in SQL. Unlike specialized functions, it allows you to craft any prompt and get intelligent…
1 min read
SQL AI Functions in Databricks: LLMs Meet SQL
Databricks SQL AI functions bring large language model capabilities directly into SQL queries. Process text, generate insights, and enrich data without…
1 min read
Natural Language Queries for Data: Beyond Simple Q&A
Natural language interfaces for data go beyond simple Q&A to enable complex analytical conversations. This guide explores advanced natural language query…
1 min read
Natural Language to SQL: Building Intelligent Query Interfaces
Natural language to SQL (NL2SQL) transforms how users interact with databases. This guide covers implementation strategies, from simple approaches to…
2 min read
March 2024 AI Recap: Claude 3, Model Evaluation, and RAG Advances
March 2024 was a transformative month for AI. Here's a comprehensive recap of the key developments and what they mean for practitioners.
1 min read
Answer Relevancy in RAG: Does the Response Address the Question?
An answer can be factually correct and grounded in context but still fail to address what was actually asked. Answer relevancy measures how well the…
1 min read
Generation Metrics for RAG: Measuring Answer Quality
While retrieval metrics measure what documents are found, generation metrics evaluate the quality of the synthesized answer. This guide covers metrics…
1 min read
RAGAS Framework: Automated RAG Evaluation
RAGAS (Retrieval Augmented Generation Assessment) is an open-source framework for evaluating RAG pipelines. This guide covers how to implement and use RAGAS…
1 min read
RAG Evaluation: Measuring Retrieval-Augmented Generation Quality
Retrieval-Augmented Generation (RAG) systems combine retrieval and generation components, each requiring specific evaluation strategies. This guide covers…
1 min read
Task-Specific Evaluation for LLMs: Beyond Generic Benchmarks
Generic benchmarks like MMLU and HumanEval don't predict performance on your specific use cases. This guide covers how to design and implement task-specific…
1 min read
HumanEval Metrics: Measuring Code Generation Quality
HumanEval is the standard benchmark for measuring LLM code generation capabilities. Understanding its methodology and limitations is essential for…
1 min read
MMLU Scores Explained: What They Really Mean
MMLU (Massive Multitask Language Understanding) is one of the most cited LLM benchmarks. Understanding what it measures and its limitations is crucial for…
1 min read
LLM Benchmarking: Understanding Model Performance
Understanding LLM benchmarks is essential for making informed model selection decisions. This guide covers the major benchmarks and how to interpret their…
1 min read
Open Source vs Proprietary LLMs: Making the Right Choice
The choice between open-source and proprietary LLMs is one of the most important architectural decisions for AI projects. Let's analyze the tradeoffs…
1 min read
Meta Llama 2 70B on Azure: Open Source Power
Meta's Llama 2 70B represents the state of the art in open-source large language models. Available on Azure AI, it offers a compelling alternative to…
1 min read
Mistral Large on Azure: Getting Started Guide
Mistral Large is now available on Azure AI, bringing one of Europe's most capable AI models to the Azure ecosystem. This guide covers deployment, usage, and…
1 min read
Azure AI Model Catalog: March 2024 Updates
The Azure AI Model Catalog continues to expand with new models and capabilities. This month brings significant updates including new foundation models and…
2 min read
Claude 3 Released: Anthropic's Most Capable AI Model Family
Today, Anthropic released Claude 3 - their most capable AI model family yet. With three models (Opus, Sonnet, and Haiku), Anthropic is directly challenging…
1 min read
Model Selection Strategies: Right-Sizing Your AI Workloads
While the AI community anticipates Claude 3's tiered model approach, it's worth examining how to right-size your AI workloads today. Understanding when to…
2 min read
Claude 2.1 vs GPT-4: A Technical Comparison While We Await Claude 3
With Claude 3 expected soon, now is a good time to compare the current state of play between Claude 2.1 and GPT-4. Let's dive into a technical comparison of…
2 min read
Anticipating Claude 3: What We Know So Far
Anthropic has been signaling that Claude 3 is on the horizon, and the AI community is buzzing with anticipation. Based on Anthropic's track record and hints…
1 min read
Semantic Caching: Intelligent Response Reuse for LLMs
1. Start conservative Higher threshold (0.95+) for critical applications 2. Tune with data Evaluate on your actual query patterns 3. Monitor quality Track…
1 min read
Caching Strategies for LLM Cost Reduction
1. Start with exact match Simple and effective 2. Add semantic caching For variable phrasing 3. Set appropriate TTL Balance freshness and savings 4. Monitor…
1 min read
Token Cost Analysis: Understanding and Managing LLM Expenses
Token cost analysis enables informed decisions about LLM usage. Track, analyze, and optimize to keep costs predictable and manageable.
1 min read
Cost Optimization for LLMs: Strategies That Work
1. Cache aggressively Identical queries don't need re processing 2. Tier your models Match model to task complexity 3. Optimize prompts Shorter prompts =…
1 min read
LLM Model Selection Criteria: A Decision Framework
Model selection should be systematic, not arbitrary. Define your requirements, score candidates objectively, and document decisions for future review.
1 min read
Model Risk Management for Large Language Models
LLMs are different beasts: unpredictable, context‑sensitive, and often opaque. Model risk management for LLMs needs to emphasise provenance, prompt…
1 min read
2023 AI Year in Review: The Year Everything Changed (Part 1)
2023 will be talked about for a long time in AI circles — the year foundation models moved from research labs into the backbone of enterprise software. From…
1 min read
LLM Benchmarking: Evaluating Models for Your Use Case
LLM benchmarking methodology matters more than the benchmark scores themselves, because the standard public benchmarks (MMLU for general knowledge…
2 min read
Open Source LLMs: Comparing Llama, Mistral, and Beyond
The open-source LLM landscape in late 2023 is richer than most enterprise teams have had time to evaluate — and the pace of model releases (Llama 2 in July…
2 min read
Mistral 7B on Azure: Exploring Open-Source LLM Deployment
Mistral 7B — released by Mistral AI on September 27, 2023 with a permissive Apache 2.0 licence — is the open-source model that forced a recalibration of…
1 min read
Azure Model Catalog: Choosing the Right AI Model
The Azure Model Catalog (available in Azure AI Studio and Azure ML) is where Microsoft is building the answer to the model selection question — "which…
1 min read
Azure AI Studio: Building Enterprise AI Applications
Azure AI Studio arrived at Ignite 2023 as a significantly expanded platform — and "expanded" is the right word, because it builds on Azure ML Studio's…
5 min read
LLM Application Testing: Strategies and Frameworks
Testing LLM applications is a problem that doesn't have a satisfying general-purpose solution yet, and that's worth acknowledging before diving into…
1 min read
RAG Improvements: Advanced Retrieval Techniques
Advanced techniques for improving Retrieval-Augmented Generation systems for better accuracy and relevance.
1 min read
Hallucination Mitigation Strategies for LLM Applications
Comprehensive strategies for detecting and mitigating hallucinations in LLM-generated content.
1 min read
Prompt Injection Defense: Protecting LLM Applications
Comprehensive strategies for defending against prompt injection attacks in LLM applications.
1 min read
AI Safety Fundamentals for LLM Applications
Essential AI safety concepts and practices for building responsible LLM applications.
1 min read
Human Feedback in LLM Development
Implementing human feedback systems to improve LLM application quality through user input and expert evaluation.
1 min read
Parallel Execution in LangChain: Optimizing LLM Applications
Master parallel execution patterns in LangChain to build faster, more efficient LLM applications.
1 min read
Chain Composition in LangChain: Building Complex Workflows
Learn advanced techniques for composing LangChain chains into complex, multi-step LLM workflows.
1 min read
Understanding Runnable Interfaces in LangChain
Deep dive into LangChain's Runnable interface and its implementations for building flexible LLM pipelines.
1 min read
LCEL Design Patterns for Production LLM Applications
Advanced LCEL patterns for building robust, scalable LLM applications in production environments.
1 min read
LangChain Expression Language (LCEL): A Practical Guide
Understanding LangChain Expression Language for building composable LLM applications with clean, declarative syntax.
1 min read
Vector Store Integrations: Choosing and Using Vector Databases
This concludes our August 2023 series on LLM optimization and vector stores.
1 min read
LangChain Updates: New Features and Improvements
LangChain's velocity in 2023 has been remarkable and occasionally destabilising — the 0.0.x series has introduced breaking API changes in point releases…
1 min read
Memory Management for LLM Applications
Tomorrow we'll explore LangChain updates and new features. LangChain Memory Vector Stores Conversational Memory Patterns
1 min read
Conversation Summarization: Managing Long Chat Histories
Conversation summarisation is the practical solution to the context window problem for long-running chat applications. The approach: when the accumulated…
1 min read
Context Caching for LLM Applications
Tomorrow we'll explore conversation summarization techniques. Redis Caching Semantic Search LLM Caching Patterns
1 min read
Semantic Compression: Preserving Meaning with Fewer Tokens
Tomorrow we'll explore context caching strategies. Text Summarization Survey Sentence Transformers ROUGE Score
1 min read
Prompt Compression: Reducing Token Usage
Tomorrow we'll explore semantic compression techniques in more depth. LLMLingua Prompt Compression Paper Token Optimization Guide
1 min read
Batching Strategies for LLM Inference
Tomorrow we'll explore GPU optimization techniques. Continuous Batching Paper vLLM Batching Triton Inference Server
1 min read
Efficient Inference: Optimizing LLM Response Times
Tomorrow we'll explore batching strategies in detail. vLLM TensorRT LLM Speculative Decoding Paper
1 min read
Parameter-Efficient Fine-Tuning: A Comprehensive Guide
Tomorrow we'll explore PEFT libraries and their practical usage. PEFT Library Documentation Adapter Transformers PEFT Methods Survey
1 min read
LoRA and QLoRA: Efficient Fine-Tuning for Large Language Models
Tomorrow we'll explore parameter efficient fine tuning methods in more depth. LoRA Paper QLoRA Paper PEFT Library Hugging Face LoRA Guide
1 min read
LLM Fine-Tuning Strategies: When and How to Customize Models
Fine-tuning is the right answer to fewer questions than the hype suggests, and I want to be precise about when it actually makes sense before diving into…
1 min read
Token Budgeting: Cost-Effective LLM Applications
Token costs for GPT-4 in mid-2023 are real enough to design around: roughly $0.03 per 1K input tokens and $0.06 per 1K output tokens for the 8K context…
1 min read
Context Pruning: Managing Token Limits in LLM Applications
Effective context pruning ensures your LLM applications work reliably. Tomorrow, I will cover token budgeting strategies.
1 min read
Building AI Agents: From Chatbots to Autonomous Assistants
Function calling, released last week with the 0613 model versions, gives AI agents a proper foundation — and I've been rebuilding some agent prototypes to…
1 min read
Prompt Flow General Availability: Building Production LLM Applications
Prompt Flow provides the foundation for building production-grade LLM applications. Tomorrow, I will cover Azure Machine Learning updates from Build 2023.
1 min read
Building AI Agents: From Chatbots to Autonomous Systems
AI agents represent the next evolution of AI systems. By combining planning, tool use, and memory, they can autonomously accomplish complex goals while…
1 min read
LLM Caching Strategies: Reducing Costs and Latency
LLM caching strategies are essential for production systems. By combining exact matching, semantic similarity, and intelligent invalidation, you can…
1 min read
AI Orchestration Patterns: Coordinating Multi-Model Systems
AI orchestration patterns enable building sophisticated AI systems from modular components. The key is designing for reliability, observability, and…
1 min read
Entity Extraction at Scale with LLMs
Entity extraction at scale transforms unstructured text into structured knowledge. Combining LLM intelligence with distributed processing enables insights…
1 min read
Feature Engineering with LLMs: AI-Powered Feature Creation
LLM-powered feature engineering unlocks value from unstructured data. Combine semantic understanding with traditional ML for more powerful predictive models.
6 min read
Completions Now, Chat Later: Keeping Azure OpenAI Code Portable
Azure OpenAI is GA with a Completions API only, and ChatGPT is coming soon. How to decide what needs chat and keep your prompt code ready for the change.
7 min read
ChatGPT-Style Chat on Azure OpenAI Without a Chat API
There is no ChatGPT API yet. Here is how to build multi-turn chat on Azure OpenAI's Completions API: transcripts, state, token budgets and streaming.
1 min read
Building LLM Applications with Azure OpenAI: Patterns and Practices
When building applications with GPT 3 and other LLMs, you need to think about: Prompt design and management Chaining multiple LLM calls Testing and…