Large Language Models (LLMs) are remarkable reasoning engines, but they have two well-known operational limits in production:

  1. A fixed knowledge cutoff date.
  2. Zero visibility into your private internal systems and codebase.

Fine-tuning a foundational model to inject internal domain knowledge is expensive, time-consuming, and immediately outdated the moment your documentation changes.

The battle-tested design pattern for solving this is Retrieval-Augmented Generation (RAG). RAG treats the LLM as an on-demand reasoning engine rather than a static database: when a user asks a question, the system queries a vector store for relevant excerpts, injects those grounded passages into the prompt context, and instructs the model to answer based strictly on the retrieved evidence.

In this guide, we'll build an end-to-end RAG service using .NET 10, Microsoft's unified Microsoft.Extensions.AI abstractions, and AWS Bedrock.

Architecture Overview 

flowchart TD subgraph Ingestion["1. Ingestion Pipeline (Offline / Background)"] Doc["Raw Document (Markdown / Text)"] --> Chunking["Sliding-Window Chunking (500 chars / 50 overlap)"] Chunking --> Titan["AWS Bedrock: Amazon Titan Text Embeddings G1"] Titan --> OSIndex[("OpenSearch: k-NN Vector Index (1536-dim)")] end subgraph Query["2. Query & Generation Pipeline (Online / Synchronous)"] User["Client Request"] -->|POST /api/rag/ask| API["ASP.NET Core Minimal API"] API -->|1. Generate query vector| BedrockEmb["AWS Bedrock (Titan Embeddings)"] BedrockEmb -->|2. 1536d Float Array| API API -->|3. k-NN similarity search| OSSearch[("OpenSearch: Cosine Distance Search")] OSSearch -->|4. Top-k Relevant Chunks| API API -->|5. Guardrail System Prompt + Grounded Context| Claude["AWS Bedrock: Claude 3.5 Sonnet"] Claude -->|6. Grounded Answer + Source Attributions| API API -->|7. JSON Response| User end

Core Components

  • Runtime: .NET 10 Minimal APIs.
  • AI Abstraction Layer: Microsoft.Extensions.AI (IEmbeddingGenerator and IChatClient) for decoupled, provider-agnostic interfaces.
  • Vector Embeddings: Amazon Titan Text Embeddings G1 (amazon.titan-embed-text-v1) generating 1,536-dimensional dense vectors.
  • Reasoning Model: Anthropic Claude 3.5 Sonnet hosted on AWS Bedrock Runtime.
  • Vector Store: OpenSearch (configured with an HNSW k-NN index using cosine similarity).

1. AWS Infrastructure & Console Setup

Before touching code, your AWS environment needs access to Bedrock models and a functional vector database.

Step 1.1: Request Bedrock Model Access

Foundational models in Bedrock are locked by default until requested in the console.

  1. Open the AWS Management Console and switch to us-east-1 (N. Virginia).
  2. Search for Amazon Bedrock.
  3. In the left navigation menu, scroll down to Model access and click Modify model access (or Enable specific models).
  4. Select the two models we need:
    • Amazon: Titan Text Embeddings G1 - Text (amazon.titan-embed-text-v1).
    • Anthropic: Claude 3.5 Sonnet. > [!NOTE] > Anthropic models require a brief one-time use-case declaration (company name, URL, and high-level purpose). Approval is typically granted automatically within seconds.
  5. Click Next and Submit. Confirm that the status switches to Access granted.

Step 1.2: Set Up IAM Credentials & Local AWS CLI

The official .NET AWS SDK resolves credentials via the standard AWS credential chain (environment variables $\rightarrow$ ~/.aws/credentials $\rightarrow$ IAM roles).

  1. In the AWS Console, open IAM $\rightarrow$ Users $\rightarrow$ Create user (e.g. dotnet-rag-dev).
  2. Attach the AmazonBedrockFullAccess managed policy (or a scoped policy granting bedrock:InvokeModel).
  3. Under the Security credentials tab, generate an Access Key for the CLI.
  4. Configure your local environment: 
 aws configure

Provide your AWS Access Key ID, AWS Secret Access Key, and set the default region to us-east-1.

Step 1.3: Choose Your Vector Store (The Cost Factor)

You have two viable routes for the vector database:

Option A: Local Docker Container (Recommended for Local Dev & Zero Cost)

Amazon OpenSearch Serverless has a minimum allocation of 2 OpenSearch Compute Units (OCUs) — 1 for search, 1 for indexing — which costs roughly \$175/month even when completely idle. For local prototyping and testing, run OpenSearch locally with k-NN pre-installed:

docker run -d -p 9200:9200 -p 9600:9600 \ -e "discovery.type=single-node" \ -e "plugins.security.disabled=true" \ --name opensearch-rag \ opensearchproject/opensearch:2.11.0

Option B: Amazon OpenSearch Serverless (AOSS) in AWS

If you are deploying to an AWS environment:

  1. Navigate to Amazon OpenSearch Service $\rightarrow$ Serverless $\rightarrow$ Collections $\rightarrow$ Create collection.
  2. Name it rag-knowledge-base and select Vector search.
  3. Create the three required policies:
    • Encryption policy: AWS-owned KMS key.
    • Network policy: Public access enabled for the collection endpoint.
    • Data access policy: Map your IAM user/role with index management permissions (aoss:*).
  4. Note the generated collection endpoint URL (https://<collection-id>.us-east-1.aoss.amazonaws.com). > [!WARNING] > OpenSearch Serverless strictly requires all HTTP traffic to be signed using AWS SigV4. Direct unauthenticated HTTP requests will be rejected with HTTP 403 Forbidden.

2. Project Setup & Dependencies

Initialize a new ASP.NET Core Web API project:

dotnet new webapi -n DotnetRagAws cd DotnetRagAws

Install the AWS Bedrock SDK, the Microsoft.Extensions.AI package, and the OpenSearch client:

# Microsoft AI abstractions dotnet add package Microsoft.Extensions.AI dotnet add package Microsoft.Extensions.AI.Abstractions # AWS Bedrock Runtime client dotnet add package AWSSDK.BedrockRuntime # OpenSearch .NET client dotnet add package OpenSearch.Client

Ensure your appsettings.json points to your vector store:

{ "Logging": { "LogLevel": { "Default": "Information", "Microsoft.AspNetCore": "Warning" } }, "AllowedHosts": "*", "OpenSearch": { "Endpoint": "http://localhost:9200" } }

3. Implementation

3.1. Document Chunk Model (DocumentChunk.cs)

This entity represents an indexed passage stored in OpenSearch:

using System.Text.Json.Serialization; public class DocumentChunk { [JsonPropertyName("id")] public string Id { get; set; } = Guid.NewGuid().ToString(); [JsonPropertyName("content")] public string Content { get; set; } = string.Empty; [JsonPropertyName("source")] public string Source { get; set; } = string.Empty; [JsonPropertyName("embedding")] public float[] Embedding { get; set; } = Array.Empty<float>(); }

3.2. Embedding Generator Adapter (BedrockEmbeddingGenerator.cs)

We adapt Amazon Titan Text Embeddings to the standard IEmbeddingGenerator<string, Embedding<float>> interface. Titan produces a 1,536-dimensional floating-point array:

using System.Text.Json; using System.Text.Json.Serialization; using Amazon.BedrockRuntime; using Amazon.BedrockRuntime.Model; using Microsoft.Extensions.AI; public class BedrockEmbeddingGenerator : IEmbeddingGenerator<string, Embedding<float>> { private readonly IAmazonBedrockRuntime _bedrockClient; private readonly string _modelId; public BedrockEmbeddingGenerator(IAmazonBedrockRuntime bedrockClient, string modelId = "amazon.titan-embed-text-v1") { _bedrockClient = bedrockClient; _modelId = modelId; } public async Task<GeneratedEmbeddings<Embedding<float>>> GenerateAsync( IEnumerable<string> values, EmbeddingGenerationOptions? options = null, CancellationToken cancellationToken = default) { var embeddings = new List<Embedding<float>>(); foreach (var text in values) { var payload = JsonSerializer.Serialize(new { inputText = text }); var request = new InvokeModelRequest { ModelId = _modelId, ContentType = "application/json", Accept = "application/json", Body = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(payload)) }; var response = await _bedrockClient.InvokeModelAsync(request, cancellationToken); using var reader = new StreamReader(response.Body); var responseBody = await reader.ReadToEndAsync(cancellationToken); var result = JsonSerializer.Deserialize<TitanEmbeddingResponse>(responseBody); if (result?.Embedding != null) { embeddings.Add(new Embedding<float>(result.Embedding)); } } return new GeneratedEmbeddings<Embedding<float>>(embeddings); } public EmbeddingGeneratorMetadata Metadata => new("BedrockEmbeddingGenerator"); public object? GetService(Type serviceType, object? serviceKey = null) => null; public void Dispose() { } private sealed record TitanEmbeddingResponse( [property: JsonPropertyName("embedding")] float[] Embedding ); }

3.3. OpenSearch Vector Store (OpenSearchVectorStore.cs)

This service initializes an HNSW k-NN index configured for 1,536 dimensions with cosine similarity, and runs vector nearest-neighbor queries:

using System.Net.Http.Json; using System.Text.Json; using OpenSearch.Client; public class OpenSearchVectorStore { private readonly IOpenSearchClient _client; private const string IndexName = "rag-knowledge-base"; public OpenSearchVectorStore(IOpenSearchClient client) { _client = client; } public async Task EnsureIndexExistsAsync(CancellationToken cancellationToken = default) { var existsResponse = await _client.Indices.ExistsAsync(IndexName, ct: cancellationToken); if (existsResponse.Exists) { return; } var createIndexResponse = await _client.Indices.CreateAsync(IndexName, c => c .Settings(s => s .Setting("index.knn", true) ) .Map<DocumentChunk>(m => m .Properties(p => p .Keyword(k => k.Name(n => n.Id)) .Text(t => t.Name(n => n.Content)) .Keyword(k => k.Name(n => n.Source)) .KnnVector(v => v .Name(n => n.Embedding) .Dimension(1536) .Method(mi => mi .Name("hnsw") .Engine("nmslib") .SpaceType("cosinesimil") ) ) ) ), cancellationToken ); if (!createIndexResponse.IsValid) { throw new InvalidOperationException($"Failed to create OpenSearch index: {createIndexResponse.DebugInformation}"); } } public async Task IndexDocumentsAsync(IEnumerable<DocumentChunk> documents, CancellationToken cancellationToken = default) { var bulkResponse = await _client.BulkAsync(b => b .Index(IndexName) .IndexMany(documents), cancellationToken ); if (!bulkResponse.IsValid) { throw new InvalidOperationException($"Failed to bulk index documents: {bulkResponse.DebugInformation}"); } } public async Task<IReadOnlyList<DocumentChunk>> SearchSimilarAsync( float[] queryEmbedding, int k = 3, CancellationToken cancellationToken = default) { var searchResponse = await _client.SearchAsync<DocumentChunk>(s => s .Index(IndexName) .Size(k) .Query(q => q .Knn(knn => knn .Field(f => f.Embedding) .Vector(queryEmbedding) .K(k) ) ), cancellationToken ); if (!searchResponse.IsValid) { throw new InvalidOperationException($"Vector search query failed: {searchResponse.DebugInformation}"); } return searchResponse.Documents.ToList(); } }

3.4. Ingestion Pipeline (IngestionService.cs)

Splits raw documentation into chunks using a sliding window with overlap, generates embeddings, and bulk-loads them into the vector store:

using Microsoft.Extensions.AI; public class IngestionService { private readonly IEmbeddingGenerator<string, Embedding<float>> _embeddingGenerator; private readonly OpenSearchVectorStore _vectorStore; public IngestionService( IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator, OpenSearchVectorStore vectorStore) { _embeddingGenerator = embeddingGenerator; _vectorStore = vectorStore; } public async Task IngestAsync(string rawText, string sourceDocument, int chunkSize = 500, int chunkOverlap = 50, CancellationToken cancellationToken = default) { await _vectorStore.EnsureIndexExistsAsync(cancellationToken); var chunks = ChunkText(rawText, chunkSize, chunkOverlap); var documentChunks = new List<DocumentChunk>(); foreach (var chunk in chunks) { var embeddings = await _embeddingGenerator.GenerateAsync(new[] { chunk }, cancellationToken: cancellationToken); var embeddingVector = embeddings.First().Vector.ToArray(); documentChunks.Add(new DocumentChunk { Content = chunk, Source = sourceDocument, Embedding = embeddingVector }); } await _vectorStore.IndexDocumentsAsync(documentChunks, cancellationToken); } private static List<string> ChunkText(string text, int chunkSize, int overlap) { var chunks = new List<string>(); if (string.IsNullOrWhiteSpace(text)) { return chunks; } var start = 0; while (start < text.Length) { var length = Math.Min(chunkSize, text.Length - start); var chunk = text.Substring(start, length).Trim(); if (!string.IsNullOrEmpty(chunk)) { chunks.Add(chunk); } if (start + length >= text.Length) { break; } start += chunkSize - overlap; } return chunks; } }

3.5. Bedrock Claude Chat Client (BedrockChatClient.cs)

Plugs Anthropic Claude 3.5 Sonnet on Bedrock into the modern IChatClient interface (GetResponseAsync returning ChatResponse):

using System.Text.Json; using System.Text.Json.Serialization; using Amazon.BedrockRuntime; using Amazon.BedrockRuntime.Model; using Microsoft.Extensions.AI; public class BedrockChatClient : IChatClient { private readonly IAmazonBedrockRuntime _bedrockClient; private readonly string _modelId; public BedrockChatClient(IAmazonBedrockRuntime bedrockClient, string modelId = "anthropic.claude-3-5-sonnet-20241022-v2:0") { _bedrockClient = bedrockClient; _modelId = modelId; } public async Task<ChatResponse> GetResponseAsync( IEnumerable<ChatMessage> chatMessages, ChatOptions? options = null, CancellationToken cancellationToken = default) { var messagesList = chatMessages.ToList(); var systemMessage = messagesList.FirstOrDefault(m => m.Role == ChatRole.System)?.Text ?? string.Empty; var conversationMessages = messagesList .Where(m => m.Role != ChatRole.System) .Select(m => new { role = m.Role == ChatRole.User ? "user" : "assistant", content = m.Text }) .ToList(); var payload = new { anthropic_version = "bedrock-2023-05-31", max_tokens = options?.MaxOutputTokens ?? 1000, system = systemMessage, messages = conversationMessages }; var request = new InvokeModelRequest { ModelId Source：https://dev.to/joaooliveiratech/building-a-resilient-rag-system-with-net-10-and-aws-bedrock-6mo
My favorites
PreviousConductor alternatives in 2026NextI built a zero-dependency, local-first Python & browser suite (SAST, contracts, invoices, tasks)

Related posts

Jensen Huang’s exit puts Nvidia’s support for open source in question

Jensen Huang’s exit puts Nvidia’s support for open source in question

TechCrunch
"Eye of Sauron " for AI agents

"Eye of Sauron " for AI agents

dev.to
I built a trading chart whose journal fills itself, on one FastAPI process and SQLite

I built a trading chart whose journal fills itself, on one FastAPI process and SQLite

dev.to
HubSpot API: how to get attachments from notes and emails (hs_attachment_ids)

HubSpot API: how to get attachments from notes and emails (hs_attachment_ids)

dev.to
When The Policy Shrinks, The Data Carries More

When The Policy Shrinks, The Data Carries More

dev.to
I built a zero-dependency, local-first Python & browser suite (SAST, contracts, invoices, tasks)

I built a zero-dependency, local-first Python & browser suite (SAST, contracts, invoices, tasks)

dev.to