1 min read
Quantization Techniques: Shrinking Models Without Losing Quality
Strategic quantization enables deploying large models on resource-constrained devices.
Quantization
5 articles
1 min read
Scalar Quantization for Vector Search: A Practical Guide
Scalar quantization maps floating point values to integers by dividing the value range into buckets: Understanding quantization error helps set…
2 min read
Binary Vectors and Quantization in Azure AI Search
With quantization, this can be reduced to 1-2 GB.
1 min read
INT8 Quantization: Practical Implementation Guide
Tomorrow we'll explore model distillation techniques. LLM.int8() Paper PyTorch Quantization ONNX Runtime Quantization
1 min read
Quantization Basics: Reducing Model Size and Improving Speed
Tomorrow we'll explore INT8 quantization in more detail. PyTorch Quantization bitsandbytes GPTQ Paper