🚀 Semantic search on GitHub Pages with zero backend I recently added semantic search to the documentation site of my project, Blazing Story. The interesting part is that the site is a Blazor WebAssembly standalone app hosted on GitHub Pages. There is no server-side code, no API, and no database. Yet users can search documentation by meaning rather than exact keywords. Here's how I built it. 🔰 What is vector search? Vector search turns data such as text or images into a "vector", an array of numbers that represents the meaning of that data. Then it computes a kind of "distance" between two vectors. The shorter the distance, the closer the meaning. Since you search by meaning, this is also called semantic search. (If you hear "distance between two vectors", you probably think of Euclidean distance. The measure I actually use in this article is **cosine similarity. I will keep using the word "distance" because it is easier to imagine, but please remember that the real measure is cosine similarity.) Unlike a simple keyword search, vector search finds data that is close in meaning. For example, a keyword search for "apple" only finds data that contains the string "apple". A vector search can also find data like "fruit" or "banana", because they are close in meaning. Turning data into a vector is called embedding. It is usually done with natural language processing (NLP). 🤩 Vector search in a real Blazor WebAssembly app Let's look at a real application. I built the documentation site of my own project, Blazing Story, as a Blazor WebAssembly standalone app, and I host it on GitHub Pages. https://blazingstory.github.io/docs/ That documentation site is where I implemented the vector search I described above. As I wrote, the site is a Blazor WebAssembly standalone app on GitHub Pages, which only serves static files. So there is no server-side code at all. Everything has to happen on the client side. To make that work, I split the job into two stages. Stage 1 happens at build time. GitHub Actions builds and deploys the documentation site on Linux (Ubuntu). As part of that build, I generate vectors from the Markdown files (.md) that are the source of the documentation, and I save those vectors into an index file that ships with the site. The tool that generates this index file is a C# console app. Stage 2 happens on the client side, in the Blazor WebAssembly app. The app loads that index file, turns the search words into a vector at search time, and searches (in other words, it computes the distance between vectors). So the vector of each target page is computed ahead of time and the browser only loads it from the index file. The vector of the search words is the only one the browser has to compute itself. With this design, a Blazor WebAssembly standalone app on a static server can find data that is close in meaning, with no server-side code at all. In this article, I will show you how to turn text into a vector, both in a C# console app and in a Blazor WebAssembly app. Note that everything here assumes English text. ⚡ Making a vector in a C# console app On the desktop, a C# app can turn text into a vector with the Microsoft.ML.OnnxRuntime and Microsoft.ML.Tokenizers NuGet packages. Thanks to these packages, the code is not so hard to write. Besides the NuGet packages, you need an NLP model in ONNX format, and the vocabulary file for the matching tokenizer. The tokenizer splits the input text into tokens, you feed those tokens to the ONNX model, and you get a vector back. Now, which model should I pick? For the Blazing Story documentation site, the language is English only. And here is a really important point. The browser has to compute the vector of the search words with the same model. If the models are different, the vectors don't match. So I wanted an ONNX model that is small enough for a browser to download. I picked a model from the MiniLM family, all-MiniLM-L6-v2. You can download its ONNX file and the vocabulary file for the tokenizer from the Hugging Face model page. https://huggingface.co/Xenova/all-MiniLM-L6-v2 Of the ONNX files on that page, the one I use is the model quantized to 8-bit integers (onnx/model_quantized.onnx). The 32-bit floating point version before quantization (onnx/model.onnx) is about 90 MB, while this quantized file is about 22 MB. That is still not small for content you load into a browser, but it is small for an NLP model, so I decided it was acceptable. I said the code is not so hard, but there are still two things you have to write yourself. The first one is about the vector you get. With Microsoft.ML.OnnxRuntime, you don't get a single vector for the input text directly (with a MiniLM model, that would be an array of 384 floats). You get one vector for each token the tokenizer produces. So I average those per-token vectors into a single n-dimensional vector that represents the whole input text. This step is called mean pooling. The second one is normalization. These vectors are used later to compute a "distance", and I normalize each vector first to keep that computation light. I will skip the details here. Thanks to this step (L2 normalization), the distance code near the end of this article stays very short. I published the sample program on GitHub, so take a look at the full code there. https://github.com/sample-by-jsakamoto/Blazor-Wasm-Embedding/tree/main/ConsoleAppEmbeddingDemo When you run this sample, it waits for text on the console. Type some English text, press Enter, and it turns the text into a vector and prints the first 10 elements of that vector. In the sample code, the embedding work is wrapped in a class named MiniLmEmbedder. You create it with the path to the ONNX model and the path to the vocabulary file. It exposes a method named Embed, which takes text and returns a vector (an array of 384 floats). Program.cs just uses that class. 🔧 Making a vector in a Blazor WebAssembly app The basic idea in a Blazor WebAssembly app is the same as on the desktop. You prepare an ONNX model and a matching tokenizer, and you combine them to turn text into a vector. However, as far as I could find, Microsoft.ML.OnnxRuntime doesn't run on a browser platform like Blazor WebAssembly. The good news is that the JavaScript ecosystem gives you plenty of options. I was a little sad that I could not do everything in C#/.NET, but on the Blazor WebAssembly side I decided to use the JavaScript library @xenova/transformers. Thanks to transformers, the code on the Blazor WebAssembly side is very short. You call the pipeline function that transformers exports, and you tell it the kind of task ("compute a vector") and the model. It gives you back a function that takes text and returns the vector of that text. You can also pass options to that function to ask for mean pooling and normalization, and it does both for you. On top of that, transformers downloads the ONNX model file and the tokenizer config files from Hugging Face, and stores them in the browser's cache storage. You don't have to write any of that yourself. On the Blazor WebAssembly side, I only call that function through JavaScript interop. That's it. Here is the sample program on GitHub. https://github.com/sample-by-jsakamoto/Blazor-Wasm-Embedding/tree/main/BlazorWasmEmbeddingDemo When you run this sample and open it in a browser, you get a text box. Type some English text, click the "Embed" button, and the page shows the first 10 elements of the vector. In wwwroot/js/embeddings.js, I import @xenova/transformers from a CDN and wrap the function so it is easy to call from Blazor WebAssembly. On the Blazor WebAssembly side, I wrote a C# service class named EmbeddingService and inject it into the Razor component. That service class is just a thin wrapper that calls embeddings.js. Now embed the same text in the C# console app and in this Blazor WebAssembly app. You will see that the two vectors are almost the same (you only see the first 10 elements, of course). They match because both sides use the same quantized ONNX model file (onnx/model_quantized.onnx) from Xenova/all-MiniLM-L6-v2. @xenova/transformers (v2) loads the quantized model by default, so it uses the same file I picked in the console app. If only one side used the 32-bit floating point version, the vectors would not match this well. They are still not exactly the same, because of tiny differences in number handling between C# and JavaScript, and differences inside the libraries. But for our purpose, comparing vectors and picking the closest one, this is close enough. 💡 Getting the distance between two vectors So both a C# app on the desktop and a Blazor WebAssembly app can turn text into a vector with a MiniLM model. Finally, here is how to get the "distance", in other words the similarity, between two vectors. This C# code does it. float Score(float[] a, float[] b) { var score = 0f; for (var d = 0; d < a.Length; d++) score += a[d] * b[d]; return score; } The vectors are already normalized, as I explained above, so the code can be this short. The "distance" you get is in the range of -1.0 to 1.0, and the closer to 1.0, the more similar the two vectors are. (You may wonder why a "distance" is not closer to 0 when two things are similar. As I mentioned at the beginning, this "distance" is not Euclidean distance. It is cosine similarity, so a bigger number means a closer match.) On the Blazor WebAssembly side, I use this code to get the "distance" (cosine similarity) between the vector of the search words and the vector of each target document. Then I pick the documents with the highest similarity, the ones whose score is closest to 1.0. That gives you a search by meaning, right in the browser. 🎉 Conclusion A C# app on the desktop and a Blazor WebAssembly app can both turn English text into a vector without much trouble. On the desktop, use the Microsoft.ML.OnnxRuntime and Microsoft.ML.Tokenizers NuGet packages. In Blazor WebAssembly, use the JavaScript library @xenova/transformers. When both sides use the same NLP model, the vectors built for your documents at build time and the vector built in the browser for the search words can be compared directly. That is how you get vector search (semantic search) in a browser, with no server-side code. To be honest, a real documentation site needs more than this. You have to split the target documents into chunks, decide the format of the index file, and so on. I skip those details here, but you can find the full code in the GitHub repository of the Blazing Story documentation site. https://github.com/BlazingStory/docs And if you want to handle a language other than English, such as Japanese, you need to do more work, such as morphological analysis, tokenizer support, and picking a model that fits that language. This article doesn't go that far. Even so, I hope you can see that a C# program on the desktop, or a Blazor WebAssembly app in a browser, can turn text into a vector and search by meaning. There is a real site doing it already, the Blazing Story documentation site, and now you know the basic idea behind it. Even a Blazor WebAssembly standalone app hosted on a plain static file server can search documents by meaning. With a little imagination, I think there are many other ways to use this. I hope you find it useful. If you try this in your own app, feel free to share how it went in the comments! 👇 ❤️ Happy coding!