{"id":110100,"date":"2026-05-01T12:00:00","date_gmt":"2026-05-01T12:00:00","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=110100"},"modified":"2026-04-23T13:01:52","modified_gmt":"2026-04-23T13:01:52","slug":"what-is-chunking-and-how-does-it-apply-to-vectors-in-sql-server-2025","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/databases\/sql-server\/what-is-chunking-and-how-does-it-apply-to-vectors-in-sql-server-2025\/","title":{"rendered":"What is chunking, and how does it apply to vectors in SQL Server 2025?"},"content":{"rendered":"\n

In this article, Greg Low explains what chunking is, why it matters for embeddings, and how SQL Server 2025 enables efficient AI-powered vector search.<\/strong><\/p>\n\n\n\n

If you’ve started to work with vector databases and looked at using text embeddings for AI search<\/a>, you might have come across the term chunking<\/em> and wondered what it relates to. In this article, I’ll explain the concept in general – and then show how it works in SQL Server 2025<\/a>.<\/p>\n\n\n\n

What is chunking?<\/h2>\n\n\n\n

To generate text embeddings, you send a body of text to an AI language model<\/a> that returns you a set of vectors. The number of values that come back (i.e., the dimensions of the vector) is determined by the model you’re using. That makes sense for product descriptions, categories, short summaries, etc, but as the text size increases, it gets more challenging. Additionally, the price you pay for using the model usually increases with the number of tokens you send.<\/p>\n\n\n\n

Chunking<\/em> is the term used to describe breaking a larger document into smaller pieces before you generate embeddings for it. Each piece (or ‘chunk’) of text is then converted to a vector. The vector represents the semantic meaning of the text. We can then store the vectors and use them to search for other text with a similar meaning. So, the chunking determines the unit of meaning you’re going to work with, and will directly impact search quality.<\/p>\n\n\n\n

When working with real documents, many are too large and too broad to embed as a single block. Think policy documents, a technical guide, contracts, a long article, transcripts…these often contain many<\/em> topics and subtopics. A vector that represents the meaning of the large document ends up like some sort of average representation of what it contains. That’s not often useful unless you’re just looking for vague similarity. What you generally want is the part of the document that’s relevant to the search.<\/p>\n\n\n\n

Issues with retrieval-augmented generation (RAG)<\/h2>\n\n\n\n

Another problem arises when working with retrieval-augmented generation (RAG)<\/a>. Relevant chunks of information are sent as part of the context when a prompt is sent to a language model.<\/p>\n\n\n\n

It’s easy for the context ‘window’ to be full of irrelevant information which makes it hard to locate the actual useful<\/em> information. The size of the context window can also be a limiting factor.<\/p>\n\n\n\n

Good chunking balances precision and context. As well as the issues with chunks being too large, if the chunks are too small, there might not be enough related information to allow for correct interpretation.<\/p>\n\n\n\n

Chunking methods, explained<\/h2>\n\n\n\n

There are different ways to chunk text.<\/p>\n\n\n

\n