{"id":111056,"date":"2026-06-01T12:00:00","date_gmt":"2026-06-01T12:00:00","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=111056"},"modified":"2026-05-28T14:47:15","modified_gmt":"2026-05-28T14:47:15","slug":"how-to-host-an-ai-text-embeddings-model-for-sql-server-using-ollama","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/databases\/sql-server\/how-to-host-an-ai-text-embeddings-model-for-sql-server-using-ollama\/","title":{"rendered":"How to host an AI text embeddings model for SQL Server using Ollama"},"content":{"rendered":"\n

When we want to use AI-based comparisons of text, via vector search in SQL Server<\/a>, we need to first generate embeddings for the text. An embedding is a numeric representation of meaning, usually represented by vectors. In this article, I’ll show you how to use Ollama<\/a> to host a server locally that can be used to generate embeddings.<\/strong><\/p>\n\n\n\n

This is the first article in Greg Low’s series ‘AI text embeddings in SQL Server: everything you need to know’.<\/a><\/em><\/strong><\/p>\n\n\n\n

What are embeddings?<\/h2>\n\n\n\n

Instead of working directly with text, images, or other rich content, an embedding represents that content as a set of numbers that capture semantic relationships learned by a model.<\/strong> This lets systems work with meaning in a mathematical way rather than relying on concepts like text matching. <\/strong><\/p>\n\n\n\n

Embeddings are the output of trained text-based AI models. One possibly surprising concept is that they’re used for similarity, as opposed to facts – and for relative closeness instead of exact matches.<\/p>\n\n\n\n

SQL Server is not designed to host or execute those models. This isn’t a limitation; it is a design choice. While it would<\/em> be possible to run code within SQL Server to generate embeddings, it just wouldn’t be a good idea. <\/p>\n\n\n\n

Note that SQL Server can already run machine learning<\/a> models directly, and<\/em> use the PREDICT<\/code> statement<\/a> to make predictions. We don’t want to be doing that with the language models we need for embeddings, though.<\/p>\n\n\n\n

Where do we host embedding models?<\/h2>\n\n\n\n

There are two common ways that systems access embedding models: cloud<\/a>-hosted services and locally-hosted services.<\/p>\n\n\n\n

Cloud-hosted embedding services<\/strong> are provided by vendors such as OpenAI<\/a> and similar platforms. In this model, text is sent to a remote API<\/a> over the network, and embeddings are returned as a service.<\/p>\n\n\n\n

These offerings are typically easy to start with, scale well, and are continuously updated by the provider. However, they introduce external dependencies and ongoing usage costs. They also increase the risk of data leaving your environment.<\/p>\n\n\n\n

Locally hosted embedding services<\/strong> run models within your own infrastructure. Tools such as Ollama<\/a> make it possible to run embedding models on a local machine or server and expose them via a local API.<\/p>\n\n\n\n

This approach provides greater control over data, avoids external network calls, and can reduce ongoing costs. At the same time, it shifts responsibility for performance, availability, updates, and resource management onto your team.<\/p>\n\n\n\n

SQL Server doesn’t care which choice you make. It has no interest in where<\/em> embeddings were generated: it only cares about having the required number of dimensions each time you retrieve a vector.<\/strong><\/p>\n\n\n\n

\n
\n
\n
\n

Fast, reliable and consistent SQL Server development…<\/h3>\n
\n …with SQL Toolbelt Essentials. 10 ingeniously simple tools for accelerating development, reducing risk, and standardizing workflows. <\/div>\n <\/div>\n Learn more & try for free<\/a>\n <\/div>\n <\/div>\n<\/section>\n\n\n

How to install Ollama<\/h2>\n\n\n\n

Ollama is easy to install. You download it from the official website<\/a> and, once installed, it (as of 2026) opens a chat window, as you can see below. For anyone who’s ever used ChatGPT<\/a>, it’s quite familiar:<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

On the right-hand-side, you can choose the model that you want to query. Ollama can automatically download and run different models but, for SQL Server use, we want to use Ollama programmatically instead.<\/strong> <\/p>\n\n\n\n

How to use Ollama programmatically<\/h2>\n\n\n\n

We can use Ollama programmatically either by using a command line interface (CLI)<\/a>, or by making REST-based calls to the service.<\/p>\n\n\n\n

For testing with SQL Server, what I like to do is:<\/p>\n\n\n

\n