Skip to content

Gateway Service - Complete Developer Documentation

Service: API Gateway
Port: 8000
Purpose: Central entry point for all backend API requests
Technology: FastAPI (Python 3.9+)
Code Location: /gateway_route/src/main.py (3604 lines, 197 endpoints)
Owner: Backend Team
Last Updated: 2025-12-26

Table of Contents

  1. Service Overview
  2. Core Architecture
  3. Configuration & Setup
  4. Authentication Flow
  5. Chatbot Management
  6. Data Processing & Upload
  7. Chat & Response Endpoints
  8. Chat History Management
  9. Payment Integration
  10. Error Handling & Monitoring
  11. Database Schemas
  12. Deployment

Service Overview

The Gateway Service is a reverse proxy that routes all client API requests to 20+ backend microservices. It handles authentication validation, CORS, logging, and provides a unified API interface to the frontend.

Key Responsibilities

  • ✅ Route requests to appropriate microservices
  • ✅ Handle CORS (Cross-Origin Resource Sharing)
  • ✅ Validate reCAPTCHA for sensitive endpoints
  • ✅ Forward multipart file uploads
  • ✅ Provide unified error responses
  • ✅ Log all requests (via Loki + DataDog)

Statistics

  • Total Endpoints: 197 (as per code outline)
  • Lines of Code: 3,604
  • Services Routed To: 20+
  • Average Response Time: 10-30ms (Gateway overhead only)

Core Architecture

Position in System

graph TB
    subgraph "Client Layer"
        A[Next.js Frontend<br/>Port 3000]
        B[Mobile App<br/>Future]
    end

    subgraph "Gateway Layer"
        C[Gateway Service<br/>Port 8000]
    end

    subgraph "Microservices"
        D1[Auth Service<br/>Port 8001]
        D2[User Service<br/>Port 8002]
        D3[Create Chatbot<br/>Port 8003]
        D4[Selection Service<br/>Port 8004]
        D5[Data Crawling<br/>Port 8005]
        D6[Response 3D<br/>Port 8011]
        D7[Response Text<br/>Port 8012]
        D8[Response Voice<br/>Port 8013]
        D9[LLM Service<br/>Port 8016]
        D10[Payment<br/>Port 8019]
        D11[...15 more services]
    end

    A --> C
    B -.-> C
    C --> D1
    C --> D2
    C --> D3
    C --> D4
    C --> D5
    C --> D6
    C --> D7
    C --> D8
    C --> D9
    C --> D10
    C --> D11

    style C fill:#FFE082

Request Flow

Standard Request:

1. Client → Gateway (8000)
2. Gateway validates CORS
3. Gateway optionally validates reCAPTCHA
4. Gateway forwards to target service
5. Target service processes & responds
6. Gateway logs request
7. Gateway returns response to client

Configuration & Setup

Environment Variables

# Database
MONGO_URI=mongodb://demo-machineagent-rc:...@...cosmos.azure.com:10255/?ssl=true&replicaSet=globaldb&...
MONGO_DB_NAME=Machine_agent_demo

# Vector Database
DEFAULT_DATABASE=milvus
MILVUS_HOST=milvus-standalone
MILVUS_PORT=19530

# Azure Storage
AZURE_SCREENSHOTS_CONTAINER_NAME=screenshots

# Security
RECAPTCHA_SECRET_KEY=your_secret_key  # Google reCAPTCHA v2

Service URLs (Internal Docker Network)

Code Location: Lines 37-56

# Authentication & User Management
AUTH_SERVICE_URL = "http://auth-service:8001"
USER_SERVICE_URL = "http://user-service:8002"

# Chatbot Creation & Configuration
CREATE_CHATBOT_SERVICE_URL = "http://create-chatbot-service:8003"
SELECTION_CHATBOT_SERVICE_URL = "http://selection-chatbot-service:8004"
DATA_CRAWLING_SERVICE_URL = "http://data-crawling-service:8005"

# Chatbot State Services
STATE_SELECTION_3D_CHATBOT_SERVICE_URL = "http://state-selection-3dchatbot-service:8006"
STATE_SELECTION_TEXT_CHATBOT_SERVICE_URL = "http://state-selection-textchatbot-service:8007"
STATE_SELECTION_VOICE_CHATBOT_SERVICE_URL = "http://state-selection-voicechatbot-service:8008"

# System & Prompts
SYSTEMPROMPT_SERVICE_URL = "http://systemprompt-service:8009"
CHATBOT_MAINTENCE_SERVICE_URL = "http://chatbot-maintenance-service:8010"

# Response Generation (CORE)
RESPONSE_3D_CHATBOT_SERVICE_URL = "http://response-3d-chatbot-service:8011"
RESPONSE_text_CHATBOT_SERVICE_URL = "http://response-text-chatbot-service:8012"
RESPONSE_voice_CHATBOT_SERVICE_URL = "http://response-voice-chatbot-service:8013"

# Supporting Services
CHAT_HISTORY_SERVICE_URL = "http://chathistory-service:8014"
DATA_COLLECTION_SERVICE_URL = "http://client-data-collection-service:8015"
LLM_MODEL_SERVICE = "http://llm-model-service:8016"
HOME_PAGE_SERVICE_URL = "http://homepage-chatbot-service:8017"
REMOTE_PHYSIO_SERVICE_URL = "http://remote-physio-service:8018"
PAYMENT_SERVICE_URL = "http://payment-service:8019"
SUPERADMIN_SERVICE_URL = "http://superadmin-service:8020"

CORS Configuration

Code Location: Lines 28-35

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # ⚠️ PRODUCTION: Should be restricted
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

Production Recommendation:

allow_origins=[
    "https://machineavatars.com",
    "https://app.machineavatars.com",
    "http://localhost:3000"  # Development only
]

Timeout & Connection Configuration

Standard Configuration (most endpoints):

timeout = httpx.Timeout(60.0)  # 60 seconds
limits = httpx.Limits(max_connections=100, max_keepalive_connections=20)

Long-Running Operations:

# PDF upload - up to 5 minutes
timeout = httpx.Timeout(300.0)

# Data crawling - up to 3.3 minutes
timeout = httpx.Timeout(200.0)

# 3D chatbot response - up to 2.5 minutes (special projects)
timeout = httpx.Timeout(150.0)

# Standard chatbot response - 100 seconds
timeout = httpx.Timeout(100.0)

Authentication Flow

1. Login (with reCAPTCHA)

Endpoint: POST /v2/login1
Code Location: Lines 82-112
Routes To: Auth Service (8001)

Purpose: Authenticate user and send OTP to email

Request:

async def proxy_login(
    email: str = Form(...),
    password: str = Form(...),
    recaptcha_token: str = Form(...)
)

Flow:

sequenceDiagram
    Client->>Gateway: POST /v2/login1<br/>(email, password, recaptcha)
    Gateway->>Google: Verify reCAPTCHA
    Google-->>Gateway: success: true/false
    alt reCAPTCHA valid
        Gateway->>Auth Service: POST /v2/login<br/>(email, password)
        Auth Service->>Cosmos DB: Find user
        Cosmos DB-->>Auth Service: User data
        Auth Service->>Email Service: Send OTP
        Auth Service-->>Gateway: {message: "OTP sent", user_id}
        Gateway-->>Client: Success response
    else reCAPTCHA invalid
        Gateway-->>Client: 400 Invalid reCAPTCHA
    end

Database Operation (Auth Service):

// Collection: users
db.users.findOne({
    email: "user@example.com"
})

// Response:
{
    "_id": ObjectId("..."),
    "user_id": "User_12345",
    "email": "user@example.com",
    "password_hash": "$2b$12$...",  // bcrypt
    "name": "John Doe",
    "subscription_date": "2024-01-15",
    "subscription_plan": "Professional",
    "created_at": "2024-01-15T10:00:00Z"
}

Response:

{
  "message": "OTP sent successfully to your email",
  "user_id": "User_12345"
}

Error Handling:

try:
    # ... reCAPTCHA + Auth Service call
except httpx.HTTPStatusError as exc:
    raise HTTPException(
        status_code=exc.response.status_code,
        detail=exc.response.text
    )

2. Verify Login OTP

Endpoint: POST /v2/verify-login-otp
Code Location: Lines 114-144
Routes To: Auth Service (8001)

Purpose: Verify OTP and return JWT token

Request:

async def proxy_verify_login_otp(
    email: str = Form(...),
    otp: str = Form(...),
    recaptcha_token: str = Form(...)
)

Response:

{
  "access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
  "token_type": "bearer",
  "user_id": "User_12345"
}

JWT Payload Structure:

{
    "sub": "User_12345",  // user_id
    "email": "user@example.com",
    "exp": 1735214400,     // Expiry (Unix timestamp)
    "iat": 1735128000      // Issued At
}

Database Operation:

// OTP validation in Auth Service
db.otp_cache.findOne({
  email: "user@example.com",
  otp: "123456",
  expires_at: { $gt: new Date() }, // Not expired
});

// On success, delete OTP
db.otp_cache.deleteOne({ email: "user@example.com" });

3. Signup

Endpoint: POST /v2/signup
Code Location: Lines 147-160
Routes To: User Service (8002)

Request:

async def proxy_signup(
    email: str = Form(...),
    password: str = Form(...),
    name: str = Form(...)
)

Database Operation:

// Collection: users
db.users.insertOne({
  user_id: "User_<timestamp>_<random>", // e.g., "User_181473"
  email: "newuser@example.com",
  password_hash: "$2b$12$...", // bcrypt with 12 rounds
  name: "Jane Doe",
  subscription_plan: "Free", // Default
  subscription_date: new Date().toISOString(),
  created_at: new Date().toISOString(),
  is_verified: false, // Requires OTP verification
});

Response:

{
  "message": "OTP sent to email for verification",
  "user_id": "User_181473"
}

4. Verify Signup OTP

Endpoint: POST /v2/verify-otp
Code Location: Lines 163-176
Routes To: User Service (8002)

Purpose: Activate account after OTP verification

Database Operation:

// Update user verification status
db.users.updateOne(
  { email: "newuser@example.com" },
  { $set: { is_verified: true } }
);

5. Password Reset Flow

Endpoints:

  • POST /v2/forgot-password (Lines 179-192)
  • POST /v2/reset-password (Lines 195-208)

Forgot Password Request:

async def proxy_forgot_password(email: str = Form(...))

Response:

{
  "message": "OTP sent to email"
}

Reset Password Request:

async def proxy_reset_password(
    email: str = Form(...),
    otp: str = Form(...),
    new_password: str = Form(...)
)

Database Operation:

// Update password
db.users.updateOne(
  { email: "user@example.com" },
  { $set: { password_hash: "$2b$12$<new_hash>" } }
);

Chatbot Management

1. Create Chatbot

Endpoint: POST /v2/create-chatbot
Code Location: Lines 225-238
Routes To: Create Chatbot Service (8003)

Purpose: Create a new chatbot project for a user

Request:

async def proxy_create_chatbot(user_id: str = Form(...))

Database Operations:

Collection: projectid_creation

db.projectid_creation.insertOne({
  _id: ObjectId("..."),
  user_id: "User_12345",
  project_id: "User-12345_Project_1", // Format: User-{user_id}_Project_{count}
  created_at: new Date().toISOString(),
  status: "created",
});

Response:

{
  "project_id": "User-12345_Project_1",
  "message": "Chatbot created successfully"
}

Project ID Generation Logic:

# In Create Chatbot Service
existing_projects = db.projectid_creation.count_documents({"user_id": user_id})
project_count = existing_projects + 1
project_id = f"User-{user_id.split('_')[1]}_Project_{project_count}"

2. Get All Chatbots

Endpoint: GET /v2/get-chatbots
Code Location: Lines 256-269
Routes To: Create Chatbot Service (8003)

Purpose: Retrieve all chatbots for a user

Request:

GET /v2/get-chatbots?user_id=User_12345

Database Query:

db.chatbot_selection
  .find({
    user_id: "User_12345",
    deleted: { $ne: true }, // Not soft-deleted
  })
  .sort({ created_at: -1 });

Response:

{
  "chatbots": [
    {
      "project_id": "User-12345_Project_1",
      "chatbot_type": "3D",
      "chatbot_purpose": "Customer Support",
      "selection_avatar": "Avatar_Emma",
      "selection_voice": "en-US-JennyNeural",
      "selection_model": "openai-35",
      "created_at": "2025-01-15T10:00:00Z",
      "data_uploaded": true,
      "embeddings_count": 245
    },
    {
      "project_id": "User-12345_Project_2",
      "chatbot_type": "Text",
      "chatbot_purpose": "Sales",
      "selection_model": "openai-4",
      "created_at": "2025-01-20T14:30:00Z",
      "data_uploaded": false
    }
  ]
}

3. Get Chatbot Status

Endpoint: GET /v2/get-chatbot-status
Code Location: Lines 272-285
Routes To: Create Chatbot Service (8003)

Purpose: Check if chatbot is ready to use (data uploaded, embeddings generated)

Request:

GET /v2/get-chatbot-status?user_id=User_12345&project_id=User-12345_Project_1

Database Query:

// Check chatbot_selection
const chatbot = db.chatbot_selection.findOne({
  user_id: "User_12345",
  project_id: "User-12345_Project_1",
});

// Check if embeddings exist in Milvus
const partition_name = "User_User-12345_Project_1"; // Sanitized
const embedding_count = milvus.get_collection_stats(
  (partitions = [partition_name])
);

Response:

{
  "status": "ready", // or "incomplete", "processing"
  "data_uploaded": true,
  "embeddings_generated": true,
  "embedding_count": 245,
  "chatbot_type": "3D",
  "model": "openai-35"
}

4. Select Chatbot Type

Endpoint: POST /v2/select-chatbot
Code Location: Lines 287-304
Routes To: Selection Chatbot Service (8004)

Purpose: Set chatbot type (3D, Text, or Voice)

Request:

async def proxy_select_chatbot(
    user_id: str = Form(...),
    project_id: str = Form(...),
    chatbot_type: str = Form(...)  # "3D", "Text", or "Voice"
)

Database Operation:

db.chatbot_selection.updateOne(
  { user_id: "User_12345", project_id: "User-12345_Project_1" },
  {
    $set: {
      chatbot_type: "3D",
      updated_at: new Date().toISOString(),
    },
  },
  { upsert: true }
);

5. Select Avatar (3D Only)

Endpoint: POST /v2/select-avatar
Code Location: Lines 402-419
Routes To: Selection Chatbot Service (8004)

Request:

async def proxy_select_avatar(
    user_id: str = Form(...),
    project_id: str = Form(...),
    selection_avatar: str = Form(...)  # e.g., "Avatar_Emma", "Avatar_David"
)

Available Avatars:

  • Avatar_Emma (Female, professional)
  • Avatar_David (Male, friendly)
  • Avatar_Sarah (Female, casual)
  • Avatar_Mark (Male, formal)

Database Operation:

db.chatbot_selection.updateOne(
  { user_id, project_id },
  { $set: { selection_avatar: "Avatar_Emma" } }
);

6. Select Voice

Endpoint: POST /v2/select-voice
Code Location: Lines 342-359
Routes To: Selection Chatbot Service (8004)

Purpose: Choose Azure Neural TTS voice

Request:

async def proxy_select_voice(
    user_id: str = Form(...),
    project_id: str = Form(...),
    selection_voice: str = Form(...)  # Azure voice ID
)

Popular Voices:

  • en-US-JennyNeural (Female, friendly)
  • en-US-GuyNeural (Male, professional)
  • en-IN-NeerjaNeural (Female, Indian accent)
  • hi-IN-SwaraNeural (Female, Hindi)

Database Operation:

db.chatbot_selection.updateOne(
  { user_id, project_id },
  { $set: { selection_voice: "en-US-JennyNeural" } }
);

7. Select LLM Model

Endpoint: POST /v2/select-model
Code Location: Lines 361-378
Routes To: Selection Chatbot Service (8004)

Purpose: Choose which LLM model to use

Request:

async def proxy_select_model(
    user_id: str = Form(...),
    project_id: str = Form(...),
    selection_model: str = Form(...)
)

Available Models:

  • openai-4 - GPT-4-0613 (best quality, expensive)
  • openai-35 - GPT-3.5 Turbo 16K (default, cost-effective)
  • openai-4o-mini - GPT-4o Mini (fast, cheap)
  • gpto1-mini - o1-mini (reasoning)
  • llama - Llama 3.3 70B
  • deepseek - DeepSeek R1
  • Gemini Flash-2.5 - Gemini 2.0
  • Claude sonnet 4 - Claude 3.5 Sonnet
  • Grok-3 - xAI Grok

Database Operation:

db.chatbot_selection.updateOne(
  { user_id, project_id },
  { $set: { selection_model: "openai-35" } }
);

8. Select Purpose

Endpoint: POST /v2/select-purpose
Code Location: Lines 306-323

Purpose: Set chatbot purpose (determines system prompt template)

Request:

async def proxy_select_purpose(
    user_id: str = Form(...),
    project_id: str = Form(...),
    chatbot_purpose: str = Form(...)
)

Chatbot Purposes:

  • Customer Support - Helpful, polite, problem-solving
  • Sales - Persuasive, product-focused
  • HR Assistant - Professional, confidential
  • Educational - Patient, explanatory
  • Custom - User-defined prompt

System Prompt Mapping:

{
    "Customer Support": "You are a helpful customer support agent. Be polite, professional, and solve problems efficiently.",
    "Sales": "You are a sales assistant. Help customers find the right products and close deals.",
    // ... etc
}

Data Processing & Upload

1. Website Crawling - Fetch Sitemap URLs

Endpoint: POST /v2/fetch-sitemap-urls
Code Location: Lines 459-482
Routes To: Data Crawling Service (8005)
Timeout: 200 seconds
Max Connections: 200

Purpose: Extract URLs from website sitemap.xml

Request:

async def proxy_fetch_sitemap_urls(
    user_id: str = Form(...),
    project_id: str = Form(...),
    domain: str = Form(...)  # e.g., "https://example.com"
)

Flow:

sequenceDiagram
    Client->>Gateway: POST /v2/fetch-sitemap-urls<br/>(domain)
    Gateway->>Data Crawling: Fetch sitemap
    Data Crawling->>Website: GET https://example.com/sitemap.xml
    Website-->>Data Crawling: XML sitemap
    Data Crawling->>Cosmos DB: Save URLs to files_secondary
    Data Crawling-->>Gateway: {urls: [...], count: 50}
    Gateway-->>Client: URL list

Database Operation:

// Collection: files_secondary
db.files_secondary.insertOne({
  user_id: "User_12345",
  project_id: "User-12345_Project_1",
  data_type: "url",
  source_url: "https://example.com",
  discovered_urls: [
    "https://example.com/about",
    "https://example.com/products",
    "https://example.com/contact",
    // ... up to 50 URLs
  ],
  created_at: new Date().toISOString(),
});

Response:

{
  "urls": ["https://example.com/about", "https://example.com/products"],
  "count": 50,
  "message": "Sitemap URLs fetched successfully"
}

2. Extract Website Content

Endpoint: POST /v2/extract-contents
Code Location: Lines 484-509
Routes To: Data Crawling Service (8005)
Timeout: 120 seconds

Purpose: Crawl selected URLs and generate embeddings (ASYNC)

Request:

async def proxy_extract_contents(
    user_id: str = Form(...),
    project_id: str = Form(...),
    domain: str = Form(...),
    urls: str = Form(...)  # JSON array as string
)

Example Request:

{
  "user_id": "User_12345",
  "project_id": "User-12345_Project_1",
  "domain": "https://example.com",
  "urls": "[\"https://example.com/about\", \"https://example.com/products\"]"
}

Response (Async Task):

{
  "task_id": "task_abc123def456",
  "message": "Content extraction started",
  "status": "in_progress"
}

Background Process:

  1. Crawl each URL (Playwright/BeautifulSoup)
  2. Extract text content
  3. Chunk text (1000 chars, 200 overlap)
  4. Generate embeddings (BAAI/bge-small-en-v1.5)
  5. Store in Milvus partition
  6. Save metadata to Cosmos DB

Database Operation (per URL):

// Collection: files
db.files.insertOne({
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    document_id: "doc_<timestamp>_<random>",
    data_type: "url",
    source_url: "https://example.com/about",
    chunks: [
        {
            chunk_index: 0,
            text: "About Us\n\nWe are a company that...",  // 1000 chars
            embedding: [0.123, -0.456, 0.789, ...],  // 384 dimensions
            chunk_length: 1000
        },
        {
            chunk_index: 1,
            text: "...company that provides excellent...",  // 1000 chars
            embedding: [0.234, -0.567, 0.890, ...],
            chunk_length: 1000
        }
        // ... more chunks
    ],
    total_chunks: 15,
    created_at: new Date().toISOString()
})

Milv us Operation:

# Each chunk → separate vector in Milvus
partition_name = "User_User-12345_Project_1"  # Sanitized project_id
collection.insert(
    partition_name=partition_name,
    data=[
        {
            "document_id": "doc_abc123",
            "user_id": "User_12345",
            "project_id": "User-12345_Project_1",
            "chunk_index": 0,
            "text": "About Us\n\nWe are...",
            "embedding": [0.123, -0.456, ...],  # 384-dim
            "data_type": "url",
            "source_url": "https://example.com/about"
        },
        # ... more chunks
    ]
)

3. Check Task Status

Endpoint: GET /v2/task-status
Code Location: Lines 512-527
Routes To: Data Crawling Service (8005)

Purpose: Monitor async crawling/embedding task

Request:

GET /v2/task-status?task_id=task_abc123def456

Response:

{
  "task_id": "task_abc123def456",
  "status": "completed", // or "in_progress", "failed"
  "progress": 100,
  "urls_processed": 50,
  "total_urls": 50,
  "embeddings_generated": 245,
  "time_elapsed": "45 seconds"
}

Error Response:

{
  "task_id": "task_abc123def456",
  "status": "failed",
  "error": "Failed to crawl https://example.com/broken-url - 404 Not Found"
}

4. PDF Upload

Endpoint: POST /v2/fetch-pdf
Code Location: Lines 1065-1094
Routes To: Client Data Collection Service (8015)
Timeout: 300 seconds (5 minutes)

Purpose: Upload and process PDF files

Request (Multipart Form):

async def proxy_fetch_pdf(
    user_id: str = Form(...),
    project_id: str = Form(...),
    files: List[UploadFile] = File(...)  # Multiple PDFs
)

Code Snippet:

# Gateway handles file forwarding
file_tuples = []
for file in files:
    content = await file.read()
    file_tuples.append(
        ("files", (file.filename, content, file.content_type))
    )

async with httpx.AsyncClient(timeout=300.0) as client:
    response = await client.post(
        f"{DATA_COLLECTION_SERVICE_URL}/v2/fetch-pdf",
        data={"user_id": user_id, "project_id": project_id},
        files=file_tuples
    )

Processing Steps:

  1. Extract text from PDF (PyPDF2)
  2. Chunk text (1000 chars, 200 overlap)
  3. Generate embeddings
  4. Store in Milvus
  5. Save metadata to Cosmos DB

Database Operation:

db.files.insertOne({
    document_id: "doc_pdf_123",
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    data_type: "pdf",
    filename: "Product_Manual.pdf",
    file_size: 2048576,  // bytes
    page_count: 25,
    chunks: [...],  // Each chunk with embedding
    total_chunks: 50,
    created_at: new Date().toISOString()
})

5. Text File Upload

Endpoint: POST /v2/fetch-text
Code Location: Lines 1097-1117
Timeout: 100 seconds

Purpose: Upload .txt files

Same processing as PDF, but simpler (no PDF parsing needed)

6. Q&A Pairs Upload

Endpoint: POST /v2/fetch-qa
Code Location: Lines 1121-1141
Timeout: 100 seconds

Purpose: Upload Q&A pairs (CSV/Excel format)

Expected Format:

Question,Answer
What is your refund policy?,"We offer 30-day returns..."
How do I contact support?,"Email us at support@..."

Processing:

  • Each Q&A pair → single embedding
  • Format: Q: {question}\nA: {answer}
  • Stored in Milvus with data_type: "qa"

Chat & Response Endpoints

1. 3D Chatbot Response

Endpoint: POST /v2/get-response-3dchatbot
Code Location: Lines 844-894
Routes To: Response 3D Chatbot Service (8011)
Timeout: 100 seconds (150s for special projects)

Purpose: Generate AI response for 3D avatar chatbot

Request:

async def proxy_get_response_3dchatbot_openai_multi(
    user_id: str = Form(...),
    project_id: str = Form(...),
    session_id: str = Form(...),  # Unique per conversation
    question: str = Form(...),
    originating_url: str = Form(...)  # Where user is chatting from
)

Special Routing (Hardcoded):

# Lines 852-874
if project_id == 'User-181473_Project_15' or project_id == 'User-181473_Project_16':
    # Route to Remote Physio Service instead
    response = await client.post(
        f"{REMOTE_PHYSIO_SERVICE_URL}/v2/get-response-physio",
        data={
            "question": question,
            "session_id": session_id,
            "avtarType": project_id
        }
    )

Response Flow:

sequenceDiagram
    Client->>Gateway: POST /v2/get-response-3dchatbot
    Gateway->>Response 3D: Forward request
    Response 3D->>Milvus: Search embeddings(query)
    Milvus-->>Response 3D: Top-5 chunks
    Response 3D->>LLM Service: Generate(context + query)
    LLM Service->>OpenAI: API call (GPT-3.5)
    OpenAI-->>LLM Service: AI response
    LLM Service-->>Response 3D: Response text
    Response 3D->>Azure TTS: Text-to-speech
    Azure TTS-->>Response 3D: Audio WAV
    Response 3D->>Azure Blob: Upload audio
    Response 3D-->>Gateway: {answer, audio_url, visemes}
    Gateway-->>Client: Complete response

Response:

{
  "answer": "Our refund policy allows returns within 30 days of purchase for a full refund...",
  "audio_url": "https://machineagentstorage.blob.core.windows.net/audio-files/response_abc123.wav",
  "visemes": [
    { "time": 0.0, "viseme": "sil" },
    { "time": 0.2, "viseme": "f" },
    { "time": 0.4, "viseme": "uw" }
    // ... lip-sync data for 3D avatar
  ],
  "similar_vectors": [
    {
      "text": "Refund Policy\n\nWe offer a 30-day money-back guarantee...",
      "source_url": "https://example.com/refunds",
      "score": 0.85
    }
  ],
  "input_tokens": 150,
  "output_tokens": 300,
  "total_tokens": 450,
  "model_used": "gpt-35-turbo-16k-0613"
}

Database Operation (in Response Service):

// 1. Fetch chatbot config
const config = db.chatbot_selection.findOne({
    user_id: "User_12345",
    project_id: "User-12345_Project_1"
})

// config contains:
// - selection_model: "openai-35"
// - selection_voice: "en-US-JennyNeural"
// - chatbot_purpose: "Customer Support"

// 2. Milvus search
partition_name = "User_User-12345_Project_1"
query_embedding = embed("What is your refund policy?")  // 384-dim
results = milvus.search(
    collection_name="embeddings",
    partition_names=[partition_name],
    data=[query_embedding],
    limit=5,
    metric_type="L2"
)

// 3. Format context for LLM
context = "\n\n".join([result.text for result in results])

// 4. Get system prompt
system_prompt = db.system_prompts_user.findOne({ user_id, project_id })
if not system_prompt:
    system_prompt = db.system_prompts_default.findOne({ purpose: "Customer Support" })

// 5. Call LLM (via LLM Service)
response = llm_service.generate(
    model="openai-35",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": f"Context:{context}\n\nQuestion: {question}"}
    ],
    temperature=0.7
)

// 6. Generate audio (Azure TTS)
audio_bytes = azure_tts.synthesize(
    text=response.answer,
    voice="en-US-JennyNeural"
)

// 7. Upload to Blob Storage
audio_url = azure_blob.upload(audio_bytes, container="audio-files")

Timeout: 100 seconds (or 150 for physio projects) to handle:

  • Milvus search: ~20ms
  • LLM API call: ~1-3 seconds
  • TTS generation: ~2-5 seconds
  • Blob upload: ~1 second
  • Total: ~5-10 seconds typically

2. Text Chatbot Response

Endpoint: POST /v2/get-response-text-chatbot
Code Location: Lines 896-919
Routes To: Response Text Chatbot Service (8012)
Timeout: 100 seconds

Simpler than 3D (no TTS, no visemes):

Response:

{
    "answer": "Our refund policy allows...",
    "similar_vectors": [...],
    "input_tokens": 150,
    "output_tokens": 300,
    "model_used": "openai-35"
}

3. Voice Chatbot Response

Endpoint: POST /v2/get-response-voice-chatbot
Code Location: Lines 946-969
Routes To: Response Voice Chatbot Service (8013)
Timeout: 100 seconds

Includes:

  • STT: User's voice → text (Whisper API)
  • LLM: Generate response
  • TTS: Text → audio

Response:

{
  "answer": "Our refund policy...",
  "audio_url": "https://..../audio.wav",
  "transcription": "What is your refund policy?", // What user said
  "model_used": "openai-35"
}

Chat History Management

1. Save Chat History

Endpoint: POST /v2/save-chat-history
Code Location: Lines 1002-1029
Routes To: Chat History Service (8014)
Timeout: 100 seconds

Purpose: Store conversation for analytics & history retrieval

Request:

async def proxy_save_chat_history(
    user_id: str = Form(...),
    project_id: str = Form(...),
    session_id: str = Form(...),
    input_prompt: str = Form(...),
    output_response: str = Form(...),
    originated_url: str = Form(...)  # Page where chat happened
)

Database Operation:

// Collection: chatbot_history
// Upsert: Add to existing session or create new
db.chatbot_history.updateOne(
  {
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    session_id: "session_abc123",
  },
  {
    $push: {
      chat_data: {
        input_prompt: "What is your refund policy?",
        output_response: "Our refund policy allows returns within 30 days...",
        input_tokens: 150,
        output_tokens: 300,
        total_tokens: 450,
        timestamp: new Date().toISOString(),
      },
    },
    $inc: { session_total_tokens: 450 },
    $set: {
      originated_url: "https://example.com/shop",
      last_interaction: new Date().toISOString(),
    },
  },
  { upsert: true }
);

Document Structure:

{
    _id: ObjectId("..."),
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    session_id: "session_abc123",
    chat_data: [
        {
            input_prompt: "Hello",
            output_response: "Hi! How can I help?",
            input_tokens: 5,
            output_tokens: 8,
            total_tokens: 13,
            timestamp: "2025-01-15T14:00:00Z"
        },
        {
            input_prompt: "What is your refund policy?",
            output_response: "Our refund policy allows...",
            input_tokens: 150,
            output_tokens: 300,
            total_tokens: 450,
            timestamp: "2025-01-15T14:01:30Z"
        }
    ],
    session_total_tokens: 463,
    originated_url: "https://example.com/shop",
    created_at: "2025-01-15T14:00:00Z",
    last_interaction: "2025-01-15T14:01:30Z",
    datetime: ISODate("2025-01-15T14:01:30Z")  // For TTL index (90 days)
}

TTL (Time To Live):

// Auto-delete after 90 days
db.chatbot_history.createIndex(
  { datetime: 1 },
  { expireAfterSeconds: 7776000 } // 90 days in seconds
);

2. Get Chat History (by Session)

Endpoint: GET /v2/get-chat-history
Code Location: Lines 1031-1041
Routes To: Chat History Service (8014)
Timeout: 30 seconds

Request:

GET /v2/get-chat-history?user_id=User_12345&project_id=User-12345_Project_1&session_id=session_abc123

Database Query:

db.chatbot_history.findOne({
  user_id: "User_12345",
  project_id: "User-12345_Project_1",
  session_id: "session_abc123",
});

Response:

{
  "session_id": "session_abc123",
  "chat_data": [
    {
      "input_prompt": "Hello",
      "output_response": "Hi! How can I help?",
      "timestamp": "2025-01-15T14:00:00Z"
    },
    {
      "input_prompt": "What is your refund policy?",
      "output_response": "Our refund policy allows...",
      "timestamp": "2025-01-15T14:01:30Z"
    }
  ],
  "total_messages": 2,
  "session_total_tokens": 463
}

3. Get All Sessions (by Project)

Endpoint: GET /v2/get-chat-history/{project_id}
Code Location: Lines 1043-1053
Timeout: 30 seconds

Purpose: Retrieve all conversation sessions for a chatbot

Request:

GET /v2/get-chat-history/User-12345_Project_1?user_id=User_12345

Database Query:

db.chatbot_history
  .find({
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
  })
  .sort({ last_interaction: -1 });

Response:

{
  "project_id": "User-12345_Project_1",
  "sessions": [
    {
      "session_id": "session_abc123",
      "message_count": 5,
      "total_tokens": 1200,
      "last_interaction": "2025-01-15T14:05:00Z",
      "originated_url": "https://example.com/shop"
    },
    {
      "session_id": "session_def456",
      "message_count": 3,
      "total_tokens": 800,
      "last_interaction": "2025-01-14T10:30:00Z",
      "originated_url": "https://example.com/support"
    }
  ],
  "total_sessions": 2
}

4. Delete Chat History

Endpoint: DELETE /delete-chatbot-history/{project_id}
Code Location: Lines 1056-1063
Timeout: 30 seconds

Purpose: Delete all conversations for a chatbot (when chatbot deleted)

Request:

DELETE /delete-chatbot-history/User-12345_Project_1

Database Operation:

db.chatbot_history.deleteMany({
  project_id: "User-12345_Project_1",
});

Payment Integration

Razorpay Payment Flow

Gateway has payment endpoints (not shown in excerpt, but present in full code)

Typical Endpoints:

  • POST /create-payment-order - Create Razorpay order
  • POST /verify-payment - Verify payment signature
  • POST /webhook - Handle Razorpay webhooks

Payment Order Creation:

# Routes to Payment Service (8019)
@app.post("/create-payment-order")
async def proxy_create_payment_order(
    user_id: str = Form(...),
    amount: int = Form(...),  # in paisa (₹299 = 29900)
    plan: str = Form(...)     # "Starter", "Professional", etc.
)

Database Operation:

// Collection: payment_orders (example)
db.payment_orders.insertOne({
  user_id: "User_12345",
  order_id: "order_abc123", // Razorpay order ID
  amount: 29900, // ₹299
  currency: "INR",
  plan: "Professional",
  status: "created",
  created_at: new Date().toISOString(),
});

Payment Verification:

// After successful payment
db.users.updateOne(
  { user_id: "User_12345" },
  {
    $set: {
      subscription_plan: "Professional",
      subscription_date: new Date().toISOString(),
      subscription_expiry: new Date(Date.now() + 30 * 24 * 60 * 60 * 1000), // 30 days
    },
  }
);

db.payment_orders.updateOne(
  { order_id: "order_abc123" },
  { $set: { status: "paid", paid_at: new Date().toISOString() } }
);

Error Handling & Monitoring

Standard Error Pattern

All endpoints follow this pattern:

async with httpx.AsyncClient(timeout=timeout, limits=limits) as client:
    try:
        response = await client.post(...)
        response.raise_for_status()  # Raises HTTPStatusError if 4xx/5xx
        return response.json()
    except httpx.HTTPStatusError as exc:
        # Forward downstream service error
        raise HTTPException(
            status_code=exc.response.status_code,
            detail=exc.response.text
        )
    except httpx.RequestError as exc:
        # Network/timeout error
        raise HTTPException(
            status_code=504,  # Gateway Timeout
            detail=f"Request failed: {str(exc)}"
        )
    except Exception as e:
        # Unexpected error
        raise HTTPException(
            status_code=500,
            detail=str(e)
        )

Error Response Format:

{
  "detail": "Error message from downstream service or gateway"
}

Logging

Code Location: Lines 17-22

try:
    from .logger import logger
except ImportError:
    import logger as logger_module
    logger = logger_module.logger

# Logging examples
logger.info(f"Proxying login request for email: {email}")
logger.info(f"✓ Login proxy successful for {email}")
logger.error(f"✗ Login proxy failed for {email}: {exc.response.status_code}")

Log Format:

[2025-01-15 14:00:00] INFO - Proxying login request for email: user@example.com
[2025-01-15 14:00:01] INFO - ✓ Login proxy successful for user@example.com

Integration:

  • Loki: Centralized log aggregation (Docker labels: logging=loki)
  • DataDog: APM and request tracing

Database Schemas

Collections Used (via downstream services)

1. users

{
    _id: ObjectId("..."),
    user_id: "User_12345",
    email: "user@example.com",
    password_hash: "$2b$12$...",
    name: "John Doe",
    subscription_plan: "Professional",
    subscription_date: "2025-01-15T00:00:00Z",
    subscription_expiry: "2025-02-15T00:00:00Z",
    is_verified: true,
    created_at: "2025-01-15T10:00:00Z"
}

2. chatbot_selection

{
    _id: ObjectId("..."),
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    chatbot_type: "3D",  // "3D", "Text", "Voice"
    chatbot_purpose: "Customer Support",
    selection_avatar: "Avatar_Emma",
    selection_voice: "en-US-JennyNeural",
    selection_model: "openai-35",
    hidden_name: "Support Bot",
    title_name: "Emma - Customer Support",
    domain: "https://example.com",
    created_at: "2025-01-15T10:00:00Z",
    updated_at: "2025-01-15T14:30:00Z",
    deleted: false  // Soft delete flag
}

3. projectid_creation

{
    _id: ObjectId("..."),
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    created_at: "2025-01-15T10:00:00Z",
    status: "active"
}

4. chatbot_history

{
    _id: ObjectId("..."),
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    session_id: "session_abc123",
    chat_data: [
        {
            input_prompt: "What is your refund policy?",
            enhanced_question: "refund policy",  // Preprocessed
            output_response: "Our refund policy allows...",
            similar_vectors: [...],  // RAG chunks used
            input_tokens: 150,
            output_tokens: 300,
            total_tokens: 450,
            model_used: "gpt-35-turbo-16k-0613",
            timestamp: "2025-01-15T14:01:30Z"
        }
    ],
    session_total_tokens: 450,
    originated_url: "https://example.com/shop",
    created_at: "2025-01-15T14:00:00Z",
    last_interaction: "2025-01-15T14:01:30Z",
    datetime: ISODate("2025-01-15T14:01:30Z")  // TTL index
}

5. files (with embeddings)

{
    _id: ObjectId("..."),
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    document_id: "doc_abc123",
    data_type: "pdf",  // "pdf", "url", "text", "qa"
    filename: "Product_Manual.pdf",
    source_url: null,  // For URLs only
    chunks: [
        {
            chunk_index: 0,
            text: "Product Manual\n\nWelcome to...",
            embedding: [0.123, -0.456, 0.789, ...],  // 384 floats
            chunk_length: 1000
        }
        // ... more chunks
    ],
    total_chunks: 50,
    file_size: 2048576,  // bytes
    page_count: 25,      // For PDFs
    created_at: "2025-01-15T12:00:00Z"
}

6. files_secondary (without embeddings, for URL management)

{
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    data_type: "url",
    source_url: "https://example.com",
    discovered_urls: [
        "https://example.com/about",
        "https://example.com/products"
    ],
    url_count: 50,
    created_at: "2025-01-15T13:00:00Z"
}

7. system_prompts_user (custom prompts)

{
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    prompt_text: "You are Emma, a customer support specialist for our e-commerce store...",
    created_at: "2025-01-15T11:00:00Z"
}

8. system_prompts_default (templates)

{
    purpose: "Customer Support",
    prompt_text: "You are a helpful customer support agent. Be polite, professional...",
    created_at: "2024-01-01T00:00:00Z"
}

9. guardrails (safety rules)

{
    user_id: "User_12345",
    project_id: "User-12345_Project_1",
    prohibited_topics: ["politics", "religion"],
    prohibited_keywords: ["competitor_name_A"],
    custom_rules: [
        "Never discuss pricing without manager approval",
        "Always redirect legal questions to compliance"
    ],
    created_at: "2025-01-15T11:30:00Z"
}

Milvus Collection: embeddings

# Collection schema
{
    "id": INT64,  # Auto-increment primary key
    "document_id": VARCHAR(100),
    "user_id": VARCHAR(100),
    "project_id": VARCHAR(100),  # Partition key
    "chunk_index": INT32,
    "text": VARCHAR(2000),
    "embedding": FLOAT_VECTOR(384),  # BAAI/bge-small-en-v1.5
    "data_type": VARCHAR(50),  # "pdf", "url", "text", "qa"
    "source_url": VARCHAR(500),
    "created_at": VARCHAR(100)
}

# Index
{
    "index_type": "IVF_FLAT",
    "metric_type": "L2",
    "params": {"nlist": 128}
}

# Search parameters
{
    "metric_type": "L2",
    "params": {"nprobe": 10}
}

# Partitions (one per chatbot)
"User_User-12345_Project_1"  # Sanitized project_id
"User_User-12345_Project_2"
# ... etc

Deployment

Docker Configuration

Dockerfile (/gateway_route/Dockerfile):

FROM python:3.9-slim

WORKDIR /app

# Copy requirements
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy source code
COPY src/ .

# Expose port
EXPOSE 8000

# Start server
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

Docker Compose (docker-compose_prod.yaml):

gateway:
  build:
    context: ./gateway_route
  container_name: gateway
  ports:
    - "8000:8000"
  depends_on:
    - auth-service
  networks:
    - app-network
  labels:
    - "logging=loki"
    - "app=machine-agent-app"
  environment:
    - MONGO_URI=${MONGO_URI}
    - MONGO_DB_NAME=Machine_agent_demo
    - MILVUS_HOST=milvus-standalone
    - MILVUS_PORT=19530
    - RECAPTCHA_SECRET_KEY=${RECAPTCHA_SECRET_KEY}
  restart: always

Requirements.txt

fastapi==0.100.0
uvicorn[standard]==0.23.0
httpx==0.24.1
python-multipart==0.0.6
pydantic==2.0.0
azure-storage-blob==12.17.0

Health Check

Endpoint: GET /health (if implemented)

@app.get("/health")
async def health_check():
    return {
        "status": "healthy",
        "service": "gateway",
        "version": "1.0.0",
        "uptime_seconds": time.time() - START_TIME
    }

Performance Metrics

Operation Gateway Overhead Total Latency (with downstream)
Simple proxy (login) 5-10ms 50-100ms
File upload proxy 10-20ms 5-300 seconds
Chat response proxy 10-15ms 1-5 seconds
History retrieval 5-10ms 20-50ms

Bottlenecks:

  • Gateway itself is not the bottleneck (< 20ms overhead)
  • Downstream services (LLM, TTS, Milvus) determine total latency

Last Updated: 2025-12-26
Code Version: gateway_route/src/main.py (3604 lines)
Total Endpoints: 197
Review Cycle: Monthly

"The Gateway: Where 197 endpoints become one unified API."