Most tutorials teach you how to build a simple AI chatbot. But building an AI agent that can take real actions, use tools, and maintain state in production requires a completely different architecture. Here is how I built one.

Whe
n I set out to build my AI Career Agency app, I quickly realized that standard LLM wrappers (like a simple ChatGPT clone) were not enough.

I needed an agent. I needed an AI that could read a job description, decide if it was a good fit, use a tool to fetch my database records, and generate a tailored cover letter.

To achieve this, I used FastAPI for the backend, Google Gemini for the reasoning engine, and SQLite for state management.

##Here is the complete architectural breakdown of how to build a production-ready AI agent.##

The Architecture: From Chatbot to Agent

A standard chatbot is stateless and relies entirely on the prompt. An agent is stateful and uses tools.

In our architecture, the flow looks like this:

  1. The user sends a query to our FastAPI endpoint.
  2. FastAPI retrieves the conversation history from SQLite.
  3. The history and the new query are sent to the Gemini API.
  4. If Gemini determines it needs external data (e.g., "What jobs are in my database?"), it uses Function Calling to trigger a Python function in our backend.
  5. FastAPI executes the function, returns the data to Gemini, and Gemini generates the final answer.
  6. The new conversation is saved back to SQLite.

Here is how to implement this step-by-step.

Step 1: Setting up the Environment

First, we need our dependencies. We will use fastapi, uvicorn, and the official Google GenAI SDK.

pip install fastapi uvicorn google-genai python-dotenv sqlite3

Create a .env file to store your Gemini API key securely. Never hardcode this in your Python file.

GEMINI_API_KEY="your_api_key_here"

Step 2: Defining the Database (State Management)

An AI agent needs memory. If you don't store the conversation history, the AI will forget what you just said. For a mobile or lightweight deployment, SQLite is perfect.

import sqlite3 def init_db(): conn = sqlite3.connect('agent_memory.db') cursor = conn.cursor() cursor.execute(''' CREATE TABLE IF NOT EXISTS chat_history ( id INTEGER PRIMARY KEY AUTOINCREMENT, session_id TEXT, role TEXT, content TEXT ) ''') conn.commit() conn.close() def save_message(session_id: str, role: str, content: str): conn = sqlite3.connect('agent_memory.db') cursor = conn.cursor() cursor.execute( "INSERT INTO chat_history (session_id, role, content) VALUES (?, ?, ?)", (session_id, role, content) ) conn.commit() conn.close() def get_history(session_id: str): conn = sqlite3.connect('agent_memory.db') cursor = conn.cursor() cursor.execute( "SELECT role, content FROM chat_history WHERE session_id = ? ORDER BY id ASC", (session_id,) ) history = cursor.fetchall() conn.close() return history

Step 3: Defining the Tools (Function Calling)

This is where the magic happens. We need to create a Python function that the AI is allowed to use.

Let's create a fake tool that checks the status of a job application in our database.

def check_job_status(company_name: str) -> str: """ Checks the application status for a specific company. """ # In a real app, this queries your SQLite database. # For this example, we use a mock response. mock_database = { "google": "Applied - Waiting for response", "airbyte": "Interview Scheduled", "smashing magazine": "Offer Received" } status = mock_database.get(company_name.lower()) if status: return f"The status for {company_name} is: {status}." else: return f"No application found for {company_name}."

Step 4: Integrating Gemini with FastAPI

Now, let's connect the Gemini SDK to our FastAPI endpoint. We will pass the check_job_status function to Gemini so it knows the tool exists.

import os from fastapi import FastAPI, HTTPException from pydantic import BaseModel from google import genai from google.genai import types from dotenv import load_dotenv load_dotenv() app = FastAPI() client = genai.Client(api_key=os.getenv("GEMINI_API_KEY")) class ChatRequest(BaseModel): session_id: str message: str @app.post("/chat") async def chat_with_agent(request: ChatRequest): # 1. Save the user's message to SQLite save_message(request.session_id, "user", request.message) # 2. Retrieve history history = get_history(request.session_id) # 3. Format history for Gemini contents = [] for role, content in history: # Gemini uses "user" and "model" roles gemini_role = "model" if role == "assistant" else "user" contents.append(types.Content(role=gemini_role, parts=[types.Part(text=content)])) try: # 4. Call Gemini with the tool defined response = client.models.generate_content( model='gemini-2.5-flash', contents=contents, config=types.GenerateContentConfig( tools=[check_job_status], # Pass the function! system_instruction="You are a helpful career assistant. Use the tool provided to check job statuses if the user asks.", temperature=0.3 ) ) # 5. Save the AI's response to SQLite ai_message = response.text save_message(request.session_id, "assistant", ai_message) return {"response": ai_message} except Exception as e: raise HTTPException(status_code=500, detail=str(e))

Step 5: The Secret Sauce (Asynchronous Streaming)

If you are building an AI agent, waiting 10 seconds for a full response feels terrible to the user. You need to stream the response chunk-by-chunk, just like ChatGPT does.

##Here is how you modify the FastAPI endpoint to stream the Gemini response:##

from fastapi.responses import StreamingResponse @app.post("/chat/stream") async def chat_with_agent_stream(request: ChatRequest): save_message(request.session_id, "user", request.message) history = get_history(request.session_id) contents = [ types.Content(role="model" if role == "assistant" else "user", parts=[types.Part(text=content)]) for role, content in history ] async def generate_response(): full_response = "" # Use the async streaming method response_stream = await client.aio.models.generate_content_stream( model='gemini-2.5-flash', contents=contents, config=types.GenerateContentConfig(tools=[check_job_status]) ) async for chunk in response_stream: if chunk.text: full_response += chunk.text # Yield the chunk to the frontend immediately yield chunk.text # Save the complete message to the database after streaming finishes save_message(request.session_id, "assistant", full_response) return StreamingResponse(generate_response(), media_type="text/event-stream")

Why This Architecture Works in Production

1. Deterministic Math: Because Gemini uses the Python tool check_job_status to fetch the data, the AI doesn't have to guess or hallucinate the status. It relies on the Python code to return the truth.

2. Stateful Memory: SQLite ensures that even if the server restarts, the user's chat history remains intact.

3. Low Latency: The StreamingResponse makes the application feel instantly fast, even if Gemini takes a few seconds to finish thinking.

Conclusion

Building an AI agent is not about writing a perfect prompt. It is about building a solid backend architecture where the LLM acts as a reasoning engine, while Python does the heavy lifting.

By combining FastAPI, Gemini, and SQLite, you can build agents that are fast, reliable, and ready for real-world users.

##Have you started building AI agents yet? Let me know your thoughts in the comments below!##