RAG Chunking Strategies: Finding the Optimal Approach
The simplest approach splits text at regular intervals with optional overlap. Semantic chunking respects content boundaries like paragraphs and sections.
39 articles
The simplest approach splits text at regular intervals with optional overlap. Semantic chunking respects content boundaries like paragraphs and sections.
Choose chunking strategy based on document structure and retrieval requirements.
Audio AI enables natural voice interactions and automated content processing. Choose the right tool for your latency and accuracy requirements.
Reliable extraction requires careful schema design, multi-pass validation, and explicit handling of uncertainty. These patterns help you build extraction…
Semantic routing compares the meaning of a user's query against a set of example utterances. When the query is semantically similar to examples for a…
In projects I've worked on, chunking decisions alone changed retrieval quality more than model choice ever did. This deep dive pulls together advanced…
Learn how to use Azure AI Language's summarization capabilities for extractive and abstractive document summarization.
Build custom NER models to extract domain-specific entities from text using Azure AI Language.
Build custom text classification models using Azure AI Language for domain-specific categorization needs.
Explore the latest updates to Azure AI Language including improved entity recognition, sentiment analysis, and text analytics capabilities.
Deep dive into the latest speech-to-text improvements in Azure AI including better accuracy, noise handling, and domain-specific recognition.
Tomorrow we'll explore context caching strategies. Text Summarization Survey Sentence Transformers ROUGE Score
Tomorrow we'll explore the Accelerate library for distributed training. Transformers Documentation Pipeline API Model Hub
Entity extraction at scale transforms unstructured text into structured knowledge. Combining LLM intelligence with distributed processing enables insights…
LLM-powered SQL generation democratizes data access. With proper validation and safety measures, it enables anyone to query databases using natural language.
Effective summarization adapts to document type, size, and audience needs. These patterns provide a foundation for production-ready summarization systems.
Simple but effective for uniform content: Respect sentence boundaries: Natural document structure: Use embeddings to find natural break points: Hierarchical…
Embeddings are dense vector representations of text where: Similar meanings are close together in vector space Different meanings are far apart…
Prompt engineering in January 2023 was the skill that separated AI features that worked in demos from AI features that worked in production. The gap: a demo…
GPT 3.5 is not a single model but a family of models with different capabilities: Model Best For Max Tokens Cost text davinci 003 Complex tasks, longer…
Skillsets enable powerful AI-driven content enrichment during the indexing process.
Semantic search dramatically improves search relevance by understanding user intent rather than just matching keywords.
Key phrase extraction enables: Automatic document tagging Quick content summarization Topic trend identification Enhanced search and discovery It's a simple…
Train models to extract domain specific entities: Entity extraction turns unstructured text into structured data: Automatically identify people,…
For content that belongs to multiple categories: Complete implementation: Category classification automates text organization: Route content to appropriate…
AI Builder's sentiment analysis capability classifies text into positive, negative, neutral, or mixed sentiment categories—the prebuilt model runs over…
Named Entity Recognition is essential for extracting structured information from unstructured text, enabling applications from search enhancement to…
Key phrase extraction is a foundational NLP capability that enables efficient processing of large text collections and powers intelligent content management…
Sentiment analysis transforms unstructured feedback into actionable insights, enabling data-driven decisions about products, services, and customer experience.
Text Analytics enables rich understanding of unstructured text, powering applications from customer feedback analysis to content recommendation systems.
Data labeling in Azure ML streamlines the process of creating high-quality training data, essential for building accurate machine learning models.
This is direct OpenAI access - enterprise Azure integration may come in the future. response = openai.Completion.create( engine="text-davinci-002"…
Cognitive Services in mid-2021 is a sprawling catalogue—vision, speech, language, decision—that can be disorienting to navigate. Today's post is the…
Azure Translator is one of the Cognitive Services that I've shipped most quietly—it tends to be a two-day integration rather than a headline feature, but…
Intents : Categories of user actions (e.g., BookFlight, GetWeather) Entities : Important data to extract (e.g., locations, dates, quantities) Utterances :…
QnA Maker simplifies building knowledge based chatbots: Easy content import from URLs, documents, and manual entry Natural language understanding for…
I've built bot projects with the Bot Framework that I'm proud of, and bot projects that I wish I could forget. The difference is almost always scope…
LUIS: teaching machines to understand human language.
A retail client this week handed me six months of customer feedback in a CSV and asked the question I get every couple of months: "what are people actually…