1 min read
Model Optimization Techniques: From Training to Deployment
Model optimization is both science and art. Start with the techniques that offer the best impact for your specific constraints.
Model Optimization
5 articles
1 min read
INT8 Quantization: Practical Implementation Guide
Tomorrow we'll explore model distillation techniques. LLM.int8() Paper PyTorch Quantization ONNX Runtime Quantization
1 min read
Quantization Basics: Reducing Model Size and Improving Speed
Tomorrow we'll explore INT8 quantization in more detail. PyTorch Quantization bitsandbytes GPTQ Paper
1 min read
Model Optimization Techniques for Production Deployment
Tomorrow we'll explore quantization basics in more detail. PyTorch Quantization Flash Attention torch.compile
1 min read
ONNX Runtime: Optimizing Model Inference
ONNX Runtime is the inference engine I reach for when a Python-trained model needs to be deployed somewhere other than a Python service — a .NET…