1 min read
Prompt Caching: Reducing Latency and Cost for Repetitive Contexts
Many applications send the same system prompt and context repeatedly: 1. Identify common prefixes System prompts, few shot examples 2. Batch similar…
Blog
Thoughts on data engineering, cloud architecture, and technology. 2205 articles and counting.
Showing 24 of 2205 posts — page 37 of 92
1 min read
Caching Strategies for LLM Cost Reduction
1. Start with exact match Simple and effective 2. Add semantic caching For variable phrasing 3. Set appropriate TTL Balance freshness and savings 4. Monitor…
1 min read
Token Cost Analysis: Understanding and Managing LLM Expenses
Token cost analysis enables informed decisions about LLM usage. Track, analyze, and optimize to keep costs predictable and manageable.
1 min read
Cost Optimization for LLMs: Strategies That Work
1. Cache aggressively Identical queries don't need re processing 2. Tier your models Match model to task complexity 3. Optimize prompts Shorter prompts =…
1 min read
LLM Model Selection Criteria: A Decision Framework
Model selection should be systematic, not arbitrary. Define your requirements, score candidates objectively, and document decisions for future review.
2 min read
LLM Routing: Intelligent Model Selection for Cost and Quality
" in query or self. has code indicators(query lower): return ModelTier.STANDARD Long queries often need more capability if word count 200: return…
1 min read
Multi-LLM Strategies: Building Resilient AI Applications
Resilience : Failover when one provider has outages Cost optimization : Route to cheaper models when possible Capability matching : Use best model for each…
1 min read
Claude vs GPT: Choosing the Right LLM for Your Application
Anthropic has hinted at Claude 3 coming soon, which promises even better performance across benchmarks. Meanwhile, OpenAI continues to iterate on GPT-4. The…
1 min read
Gemini vs GPT: Practical Comparison for Enterprise Applications
I spent time comparing Gemini and GPT in enterprise settings; below are the practical differences that should influence model choice.
7 min read
Leakage-Safe Feature Engineering with scikit-learn Pipelines
Most feature engineering bugs are statistics fitted on the wrong rows. How I keep scaling, encoding and aggregates leak-free with scikit-learn pipelines.
7 min read
Azure ML in Early 2024: What Went GA at Ignite and What to Adopt
Feature store, serverless compute, prompt flow and the model catalog went GA in Azure ML in November 2023. Here's what to adopt now and what to hold off on.
7 min read
Azure AI Search Index Projections: Keys, Deletes and Chunk Order
The edge cases of index projections in Azure AI Search: what the chunk index schema needs, how keys change on re-index, deletes, and keeping chunk order.
8 min read
Azure AI Search Index Operations: Schema Changes, Rebuilds and Aliases
Which Azure AI Search index changes work in place, which force a rebuild, and how to swap versioned indexes, load documents and check health safely.
7 min read
Tuning Hybrid Search in Azure AI Search: What RRF Lets You Change
Which hybrid search knobs Azure AI Search exposes on the 2023-11-01 API, why I avoid client-side score weighting, and how to measure changes before shipping.
7 min read
Keeping Azure AI Search Vector Indexes Small Without Compression
Azure AI Search has no built-in vector compression yet. How to estimate vector index size, read the quota, and cut it with chunking and dimension choices.
7 min read
Azure AI Search Semantic Ranker: Configuration, Thresholds and Cost
How to design semantic configurations, use reranker scores as a RAG cut-off, and decide when Azure AI Search's semantic ranker is worth paying for.
7 min read
Integrated Vectorization in Azure AI Search: What It Does and Doesn't Do
What integrated vectorization in Azure AI Search actually removes from a RAG pipeline, what it leaves behind, and when I'd still embed in my own code.
7 min read
Azure AI Search After Ignite 2023: What's GA and What's Still Preview
A January 2024 stock-take of Azure AI Search for RAG: which features are GA, which are preview, and which SDK and REST versions to build on.
8 min read
Data Activator Preview Readiness: Limits, Admin Settings and Risks
A readiness checklist for the Data Activator preview in Fabric: tenant switches, throughput and alert limits, failure modes, and when to wait for GA.
9 min read
Designing Reflex Triggers in Data Activator Without Alert Spam
How Data Activator preview triggers really fire: objects, Is vs Becomes conditions, timers, Power Automate custom actions and the throughput limits to plan for.
7 min read
Keeping KQL Fast on Streaming Data: Query Habits That Matter
The KQL habits that keep queries over streaming telemetry fast in Fabric KQL databases and Azure Data Explorer, and the shortcuts that quietly slow them down.
8 min read
Fabric Eventstream Patterns: Deciding Where Stream Logic Lives
Practical Fabric Eventstream patterns: what belongs in the no-code event processor, what belongs in KQL, and what to fix before events arrive.
6 min read
Fabric Real-Time Analytics After GA: An End-to-End Streaming Design
How Eventstream, KQL databases, Power BI and the Data Activator preview fit together in Fabric as of January 2024, and when to choose something else.
9 min read
Designing a Gold Layer and Semantic Model That Stay in Direct Lake
Design-time rules for Direct Lake in Fabric: size tables to your SKU's guardrails, move logic into Delta, avoid views, and control when models reframe.