1 min read
Cost Optimization Strategies for Azure OpenAI Deployments
Balance cost and quality by routing requests to appropriate models based on task requirements.
Token Management
5 articles
1 min read
Cost Optimization for LLMs: Strategies That Work
1. Cache aggressively Identical queries don't need re processing 2. Tier your models Match model to task complexity 3. Optimize prompts Shorter prompts =…
1 min read
Token Budgeting: Cost-Effective LLM Applications
Token costs for GPT-4 in mid-2023 are real enough to design around: roughly $0.03 per 1K input tokens and $0.06 per 1K output tokens for the 8K context…
1 min read
Context Pruning: Managing Token Limits in LLM Applications
Effective context pruning ensures your LLM applications work reliably. Tomorrow, I will cover token budgeting strategies.
8 min read
Counting and Capping Tokens in the Azure OpenAI Preview
How to count tokens with tiktoken, size max_tokens, log real usage and cap spend for Azure OpenAI completion models in the January 2023 preview.