1 min read
Retrieval-Augmented Generation: Chunking Strategies for Better Results
The best chunking strategy depends on your document types. Technical documentation benefits from header-aware chunking. Conversational content works well…
Embeddings
26 articles
1 min read
Vector Database Selection: Choosing the Right Solution for RAG
The best vector database depends on your specific requirements. Azure AI Search excels for hybrid search with integrated semantic ranking. Pinecone offers…
1 min read
Embedding Models: Choosing Between OpenAI, Azure, and Open Source
The text-embedding-ada-002 and newer text-embedding-3 models provide high-quality embeddings with minimal setup.
1 min read
Embedding Models Comparison: Choosing the Right Model for Your Use Case
Balance quality, speed, and cost based on your specific retrieval needs.
1 min read
Vector Search in Databricks: Semantic Search at Scale
Databricks Vector Search enables semantic similarity search over your lakehouse data. Build RAG applications, recommendation systems, and intelligent search…
1 min read
Cohere on Azure: Enterprise Search and RAG Solutions
Cohere's models are now available on Azure AI, offering specialized capabilities for enterprise search and retrieval-augmented generation (RAG). This guide…
1 min read
Semantic Caching: Intelligent Response Reuse for LLMs
1. Start conservative Higher threshold (0.95+) for critical applications 2. Tune with data Evaluate on your actual query patterns 3. Monitor quality Track…
7 min read
Keeping Azure AI Search Vector Indexes Small Without Compression
Azure AI Search has no built-in vector compression yet. How to estimate vector index size, read the quota, and cut it with chunking and dimension choices.
7 min read
Integrated Vectorization in Azure AI Search: What It Does and Doesn't Do
What integrated vectorization in Azure AI Search actually removes from a RAG pipeline, what it leaves behind, and when I'd still embed in my own code.
1 min read
Vector Store Integrations: Choosing and Using Vector Databases
This concludes our August 2023 series on LLM optimization and vector stores.
1 min read
Azure Cognitive Search Vector Search: Building Semantic Search Applications
Vector search enables powerful semantic search capabilities. Tomorrow, I will cover hybrid retrieval patterns in more detail.
1 min read
Semantic Search for Enterprise: Beyond Keyword Matching
Enterprise semantic search transforms how organizations find and use knowledge. By understanding meaning rather than just matching keywords, these systems…
1 min read
Vector Databases for AI Applications
Vectors (embeddings) represent meaning in high-dimensional space. Similar items have similar vectors.
1 min read
Building Semantic Search with Azure OpenAI Embeddings
Embeddings are numerical representations of text that capture semantic meaning. Similar concepts have similar vectors. Azure OpenAI provides the…
1 min read
Hybrid Search: Combining Vector and Keyword Search
A robust way to combine rankings: Adjust weights based on query characteristics: 1. Start balanced : 50/50 is often a good default 2. Tune on your data :…
1 min read
Embedding Best Practices for Production AI Systems
OpenAI Embeddings Guide MTEB Benchmark Embedding Best Practices
7 min read
RAG on Azure OpenAI: Grounding GPT-3 Answers in Your Own Documents
What retrieval augmented generation is, why it beats fine-tuning for company knowledge, and a minimal loop on Azure OpenAI completions in January 2023.
7 min read
Azure Cognitive Search and Embeddings Before Native Vector Search
Azure Cognitive Search can't search vectors in January 2023. Here's how to re-rank its keyword results with Azure OpenAI embeddings, and when not to bother.
6 min read
Qdrant 0.11 with Azure OpenAI: Point IDs, Payload Indexes and Filters
Run Qdrant 0.11 with Azure OpenAI embeddings: valid point IDs, payload indexes, filtered HNSW search, memory settings, and when Qdrant isn't the right choice.
7 min read
Milvus 2.2 with Azure OpenAI Embeddings: A Self-Hosted Start
Run Milvus 2.2 standalone, load Azure OpenAI embeddings with pymilvus, and learn the schema, index, consistency and delete rules before you self-host it.
7 min read
Weaviate 1.17 with Azure OpenAI: Bring Your Own Vectors
Run Weaviate 1.17 with embeddings from Azure OpenAI: schema choices, batch imports, filtered vector search, BM25 and the hybrid search caveats to know first.
8 min read
Pinecone With Azure OpenAI Embeddings: Design the Index First
Set up a Pinecone index for Azure OpenAI embeddings: dimensions, pod types, namespaces, metadata indexing and idempotent upserts, plus when to skip it.
8 min read
Do You Need a Vector Database Yet? ANN Indexes and Trade-offs
What a vector database adds over brute force: ANN indexes, recall, filtering and updates, and how Pinecone, Weaviate, Milvus, Qdrant and pgvector compare.
8 min read
Semantic Search Prototype: Azure OpenAI Embeddings and NumPy
Build a small semantic search prototype on Azure OpenAI embeddings and NumPy, measure it with hit rate@k, add BM25, and know when to move to a real index.
8 min read
Text Embeddings on Azure OpenAI: What They Are and When to Use Them
What text embeddings are, which Azure OpenAI embeddings models you can use in January 2023, how to compare vectors, and when embeddings are the wrong tool.
1 min read
Vector Search Preview in Azure Cognitive Search
Vector search enables powerful similarity-based retrieval that complements traditional keyword search.