Back to the AI glossary Workflows
Chunking
Divide large documents into smaller, meaningful sections.
Explanation
Chunking breaks long texts into smaller pieces (chunks) so that they can be saved as embeddings and found effectively. The chunk size affects the quality of the search.
How it works
For example, a 100-page manual is divided into paragraphs or thematic blocks of 200-500 tokens. Each chunk is saved as its own embedding.
Example
A customer service manual is broken down into individual FAQ entries so that when a question arises, the system finds exactly the relevant section.
Why it matters
Good chunking determines the quality of RAG systems. Chunks that are too large dilute the search, and chunks that are too small lose the context.