1 min read
LangChain Updates: New Features and Improvements
LangChain's velocity in 2023 has been remarkable and occasionally destabilising — the 0.0.x series has introduced breaking API changes in point releases…
Blog
Thoughts on data engineering, cloud architecture, and technology. 2205 articles and counting.
Showing 24 of 2205 posts — page 44 of 92
1 min read
Memory Management for LLM Applications
Tomorrow we'll explore LangChain updates and new features. LangChain Memory Vector Stores Conversational Memory Patterns
1 min read
Conversation Summarization: Managing Long Chat Histories
Conversation summarisation is the practical solution to the context window problem for long-running chat applications. The approach: when the accumulated…
1 min read
Context Caching for LLM Applications
Tomorrow we'll explore conversation summarization techniques. Redis Caching Semantic Search LLM Caching Patterns
1 min read
Semantic Compression: Preserving Meaning with Fewer Tokens
Tomorrow we'll explore context caching strategies. Text Summarization Survey Sentence Transformers ROUGE Score
1 min read
Prompt Compression: Reducing Token Usage
Tomorrow we'll explore semantic compression techniques in more depth. LLMLingua Prompt Compression Paper Token Optimization Guide
1 min read
Token Estimation: Predicting API Costs
Tomorrow we'll explore prompt compression techniques. tiktoken Library OpenAI Tokenizer Azure OpenAI Pricing
1 min read
Rate Limit Management for AI APIs
Tomorrow we'll explore token estimation techniques. Rate Limiting Patterns Circuit Breaker Pattern Azure OpenAI Best Practices
1 min read
Azure OpenAI Quotas: Understanding and Managing Limits
Tomorrow we'll explore rate limit management strategies. Azure OpenAI Quotas Quota Management Rate Limit Best Practices
2 min read
The Importance of a Father Figure
Throughout my life, "Father Figure" has been an evolving term. At nine, the loss of my biological father left me grappling with an abstract notion of…
1 min read
Cost Optimization for AI Workloads
AI infrastructure costs compound quickly. A few patterns I've been applying with clients to bring Azure OpenAI and Azure ML spend under control: at the API…
1 min read
Spot Instances for ML: Cost-Effective Training at Scale
Tomorrow we'll explore cost optimization strategies for AI workloads. Azure Spot VMs Checkpointing Best Practices Azure ML Cost Management
1 min read
Azure ML Compute: Choosing the Right Infrastructure
Azure ML offers three distinct compute experiences, and choosing the right one makes a significant difference to both cost and development friction. For…
1 min read
GPU Optimization for LLM Workloads
Tomorrow we'll explore Azure ML compute options. PyTorch CUDA Semantics Flash Attention NVIDIA Optimization Guide
1 min read
Batching Strategies for LLM Inference
Tomorrow we'll explore GPU optimization techniques. Continuous Batching Paper vLLM Batching Triton Inference Server
1 min read
Efficient Inference: Optimizing LLM Response Times
Tomorrow we'll explore batching strategies in detail. vLLM TensorRT LLM Speculative Decoding Paper
1 min read
Small Language Models: When Bigger Isn't Better
Small Language Models: When Bigger Isn't Better
1 min read
Knowledge Distillation: Advanced Techniques and Applications
Tomorrow we'll explore small language models. Knowledge Distillation Survey DistilBERT Paper Self Distillation
1 min read
Model Distillation: Creating Efficient Student Models
Tomorrow we'll dive deeper into knowledge distillation techniques. Distilling Knowledge in Neural Networks TinyBERT DistilBERT
1 min read
INT8 Quantization: Practical Implementation Guide
Tomorrow we'll explore model distillation techniques. LLM.int8() Paper PyTorch Quantization ONNX Runtime Quantization
1 min read
Quantization Basics: Reducing Model Size and Improving Speed
Tomorrow we'll explore INT8 quantization in more detail. PyTorch Quantization bitsandbytes GPTQ Paper
1 min read
Model Optimization Techniques for Production Deployment
Tomorrow we'll explore quantization basics in more detail. PyTorch Quantization Flash Attention torch.compile
1 min read
ONNX Runtime: Optimizing Model Inference
ONNX Runtime is the inference engine I reach for when a Python-trained model needs to be deployed somewhere other than a Python service — a .NET…
1 min read
Accelerate Library: Distributed Training Made Simple
Tomorrow we'll explore ONNX Runtime for model optimization. Accelerate Documentation Accelerate GitHub Distributed Training Guide