Advanced RAG Settings

When you add knowledge sources (documents, websites, etc.) to your Chipp app, the AI uses Retrieval-Augmented Generation (RAG) to find and use relevant information. The Advanced RAG Settings let you fine-tune this process.

How Your AI Uses Your Documents

Before diving into settings, here's how your AI actually finds and uses information from your uploaded documents.

What is RAG?

RAG (Retrieval-Augmented Generation) is how your AI "reads" your documents to answer questions. Instead of memorizing everything, the AI:

1. Searches your documents for relevant information when a user asks a question
2. Retrieves the most relevant sections
3. Generates a response using that information

Think of it like a research assistant who quickly scans your files for relevant passages before answering, rather than trying to memorize every document.

What is a "Chunk"?

When you upload a document, Chipp automatically breaks it into smaller pieces called chunks. A chunk is typically a paragraph or a few paragraphs of text.

Why chunks?

• AI models have limits on how much text they can process at once
• Smaller pieces let the AI find the exact relevant section, not the whole document
• It's faster and more accurate than searching full documents

Example: A 50-page PDF might be split into 200 chunks. When a user asks a question, the AI searches all 200 chunks and retrieves only the 5 most relevant ones.

How Search Works

When a user asks a question, Chipp converts both the question and your document chunks into mathematical representations (called "embeddings"). It then finds chunks whose meaning is closest to the question's meaning.

This is semantic search - it understands meaning, not just keywords. So "How do I cancel my subscription?" will match a chunk about "ending your membership" even if those exact words aren't used.

Standard Search vs. Hybrid Search

Standard Search (Default)

Looks at each chunk independently and finds the ones most similar to the user's question.

• Works great for: FAQ-style content, focused topics, smaller knowledge bases
• Best when: Each chunk contains self-contained answers

Hybrid Search

Combines two search strategies:

1. Chunk search: Which individual chunks match the question best?
2. Document search: Which overall documents are most relevant?

Then it boosts chunks that come from highly relevant documents, even if those chunks aren't the top individual matches.

• Works great for: Multiple documents on similar topics, long documents, broad questions
• Best when: Context from the same document matters, or you have 10+ files

Example: A user asks about "return policies." Standard search might find one great chunk from your FAQ and another from a random product page. Hybrid search recognizes that your "Returns & Refunds" document is highly relevant overall, so it prioritizes chunks from that document.

Accessing RAG Settings

1. Go to your app in the Chipp dashboard
2. Navigate to Build tab
3. Scroll to Knowledge Sources
4. Click Advanced RAG Settings to expand

Settings Reference

Relevance Threshold

Default: 0.15 | Range: 0.00 - 1.00

Controls how closely a document chunk must match the user's query to be included in the AI's context.

When to adjust:

• Lower the threshold if users report the AI doesn't find relevant information that exists in your documents
• Raise the threshold if responses include too much irrelevant information or the AI seems confused by conflicting sources

Max Chunks

Default: 5 | Range: 1 - 20

The maximum number of document chunks retrieved for each user query. Each chunk is a section of your uploaded content.

When to adjust:

• Use fewer chunks for simple Q&A where answers are typically in one place
• Use more chunks when questions require synthesizing information from multiple sources, or for complex topics with scattered information

Note: More chunks means more context for the AI, but also increases token usage and may slow responses.

Hybrid Search

Default: Off

When enabled, combines two search strategies:

1. Chunk-level search: Finds individual chunks most similar to the query
2. Document-level search: Considers which documents overall are most relevant

This helps when the best answer comes from a document that's highly relevant overall, even if no single chunk perfectly matches the query.

When to enable:

• You have multiple documents on similar topics
• Users ask broad questions that span document sections
• You notice the AI missing context from relevant documents

When to keep off:

• Simple, focused knowledge bases
• When chunk-level matching works well

Document Weight

Default: 0.30 | Range: 0.00 - 1.00 | Only visible when Hybrid Search is enabled

Controls the balance between chunk-level and document-level relevance when Hybrid Search is on.

When to adjust:

• Lower values when your documents cover very different topics and chunk relevance is most important
• Higher values when you have long documents where context from the same document is valuable

Recommended Configurations

Customer Support Bot

Relevance Threshold: 0.20
Max Chunks: 5
Hybrid Search: Off

Good for FAQ-style knowledge bases where answers are self-contained.

Research Assistant

Relevance Threshold: 0.10
Max Chunks: 10
Hybrid Search: On
Document Weight: 0.40

Retrieves more context for synthesizing comprehensive answers.

Technical Documentation

Relevance Threshold: 0.25
Max Chunks: 7
Hybrid Search: On
Document Weight: 0.30

Balances precision with coverage for technical queries.

Legal/Compliance Bot

Relevance Threshold: 0.15
Max Chunks: 8
Hybrid Search: On
Document Weight: 0.50

Ensures relevant policy documents are well-represented.

Troubleshooting

AI doesn't find information that exists in my documents

1. Lower the Relevance Threshold to 0.10
2. Increase Max Chunks to 8-10
3. Enable Hybrid Search if you have multiple related documents

Responses include irrelevant information

1. Raise the Relevance Threshold to 0.25-0.35
2. Decrease Max Chunks to 3-4
3. Review your document organization

Token usage is too high

1. Reduce Max Chunks to 3-4
2. Raise Relevance Threshold to reduce low-quality matches
3. Consider splitting large documents into focused topics

AI gives conflicting information

1. Raise Relevance Threshold to be more selective
2. If using Hybrid Search, increase Document Weight to favor cohesive document sources
3. Review documents for outdated or contradictory content

Resetting to Defaults

Click Reset to defaults in the Advanced RAG Settings panel to restore all settings to their default values. This is useful if you've experimented and want to start fresh.