Encryption¶

Section: 5-security-architecture
Document: Data Encryption
Status: Comprehensive Encryption Documentation
Audience: Security teams, compliance officers, infrastructure engineers

🎯 Overview¶

MachineAvatars implements comprehensive encryption for data at rest and in transit, ensuring all sensitive information is protected using industry-standard encryption algorithms and protocols.

Encryption Standards:

Data at Rest: AES-256 (Advanced Encryption Standard)
Data in Transit: TLS 1.3 (Transport Layer Security)
Key Management: Environment variables (migrating to Azure Key Vault)
Customer-Managed Keys: Enterprise only (planned)

🔒 Data at Rest Encryption¶

MongoDB Database Encryption¶

Provider: Azure Cosmos DB for MongoDB
Encryption: Automatic Azure-managed encryption

Encryption Details¶

Algorithm: AES-256
Key Management: Microsoft-managed keys
Coverage: All data in MongoDB collections

Collections Encrypted:

users_multichatbot_v2 - User accounts, credentials
chatbot_selections - Chatbot configurations
chatbot_history - All conversation data
files / files_secondary - Uploaded documents metadata
system_prompts_user - Custom system prompts
projectid_creation - Project metadata
organisation_data - Enterprise organization info
trash_collection_name - Soft-deleted chatbots
features_per_user - Subscription features

Azure Cosmos DB Encryption:

┌─────────────────────────────────────┐
│   Application Layer                 │
│   (Backend Services)                │
└─────────────────┬───────────────────┘
                  │ HTTPS/TLS 1.3
                  ▼
┌─────────────────────────────────────┐
│   Azure Cosmos DB API               │
│   (MongoDB-compatible)              │
└─────────────────┬───────────────────┘
                  │
                  ▼
┌─────────────────────────────────────┐
│   Encryption Layer                  │
│   AES-256 (Microsoft-managed keys)  │
└─────────────────┬───────────────────┘
                  │
                  ▼
┌─────────────────────────────────────┐
│   Physical Storage                  │
│   (Azure Data Centers - India)      │
└─────────────────────────────────────┘

Benefits:

✅ Automatic encryption (no configuration needed)
✅ Transparent to applications
✅ Zero performance impact
✅ Compliance with GDPR, DPDPA, HIPAA
✅ Azure manages key rotation

Verification:

# Azure Portal: Cosmos DB > Settings > Encryption
# Status: "Enabled (Microsoft-managed keys)"

Milvus Vector Database Encryption¶

Deployment: Azure Container Instance
Encryption: Storage-level encryption

What's Encrypted:

Vector embeddings (1536 dimensions)
Metadata fields
Index structures
Collection partitions

Encryption Method:

┌─────────────────────────────────────┐
│   Milvus Application                │
└─────────────────┬───────────────────┘
                  │
                  ▼
┌─────────────────────────────────────┐
│   Azure Blob Storage                │
│   (Persistent volume for Milvus)    │
│   AES-256 encryption (automatic)    │
└─────────────────┬───────────────────┘
                  │
                  ▼
┌─────────────────────────────────────┐
│   Azure Physical Storage            │
└─────────────────────────────────────┘

Storage Encryption:

Azure Blob Storage automatically encrypts all data
AES-256 encryption
Microsoft-managed keys
No configuration required

In-Memory Data:

⚠️ Not encrypted (standard for vector databases)
Data encrypted when written to disk
Recommendation: Use network isolation for additional security

File Storage Encryption (GridFS)¶

Storage: MongoDB GridFS
Encryption: Inherited from MongoDB encryption

File Types Stored:

PDF documents
Word files (.docx)
Excel spreadsheets (.xlsx)
PowerPoint presentations (.pptx)
Text files (.txt)
User-uploaded content

Encryption Flow:

User Upload → Backend Service → GridFS → MongoDB → AES-256 Encryption → Azure Storage

File Metadata Collection:

{
    "_id": ObjectId("..."),
    "filename": "product_catalog.pdf",
    "contentType": "application/pdf",
    "length": 2048576,  // bytes
    "uploadDate": "2025-01-15T10:00:00Z",
    "metadata": {
        "user_id": "User-123456",
        "project_id": "chatbot_abc",
        "original_name": "Product Catalog.pdf"
    }
    // File contents encrypted at rest by MongoDB
}

File Security:

✅ Encrypted at rest (AES-256)
✅ Access control via user_id/project_id
✅ No public URLs
⚠️ No client-side encryption (files decrypted in backend)

Backup Encryption¶

MongoDB Backups:

Frequency: Daily automated backups
Retention: 35 days (configurable)
Encryption: AES-256 (same as primary data)
Location: Azure Backup (geo-redundant)

Backup Process:

Primary DB (AES-256) → Azure Backup Service → AES-256 Encrypted Backup → Geo-replicated Storage

Disaster Recovery:

Encrypted backups can be restored
Point-in-time recovery (PITR) available
Encryption keys managed by Azure
No key management needed for restore

🌐 Data in Transit Encryption¶

TLS 1.3 Implementation¶

Protocol: Transport Layer Security 1.3
Coverage: All network communication

Frontend to Backend¶

HTTPS Enforcement:

// Frontend API calls
const API_URL = process.env.NEXT_PUBLIC_BACKEND_URL;
// Example: "https://api.machineavatars.com"

fetch(`${API_URL}/v2/chatbots`, {
  method: "GET",
  headers: {
    Authorization: `Bearer ${token}`,
  },
});

Requirements:

✅ HTTPS only (no HTTP allowed)
✅ TLS 1.3 or TLS 1.2 minimum
✅ Valid SSL certificate
❌ Self-signed certificates rejected

TLS Configuration (Azure App Service):

Minimum TLS Version: 1.2
Preferred: TLS 1.3
Cipher Suites: Modern, secure ciphers only
HSTS: Enabled (HTTP Strict Transport Security)

Backend to Database¶

MongoDB Connection:

# Backend services
from pymongo import MongoClient

mongo_uri = os.getenv("MONGO_URI")
# Format: "mongodb+srv://username:password@cluster.mongodb.net/?ssl=true"

client = MongoClient(mongo_uri)

Connection Security:

✅ SSL/TLS enforced (ssl=true parameter)
✅ Certificate validation
✅ Encrypted credentials in connection string
⚠️ Connection string hardcoded in some services (migrating to env vars)

TLS Handshake:

Backend Service → TLS 1.2/1.3 Handshake → Azure Cosmos DB
                 ↓
          Certificate Validation
                 ↓
          Encrypted Connection Established
                 ↓
          All Data Encrypted (AES-256 in transit)

Backend to Milvus¶

Vector Database Connection:

from pymilvus import connections

connections.connect(
    alias="default",
    host="milvus.example.com",
    port="19530",
    secure=True  # Enable TLS
)

Current Status:

⚠️ TLS not enforced (internal network)
✅ Network isolation (Azure VNet)
⏳ TLS planned for production

Recommendation:

connections.connect(
    alias="default",
    host="milvus.example.com",
    port="19530",
    secure=True,
    server_pem_path="/path/to/server.pem",  # TLS certificate
    server_name="milvus.example.com"
)

Backend to Azure Services¶

1. Azure Communication Email:

from azure.communication.email import EmailClient

email_client = EmailClient(
    endpoint="https://mailing-sevice.india.communication.azure.com/",
    credential="..." # Access key
)
# All requests over HTTPS (TLS 1.2+)

2. Azure OpenAI:

import openai

openai.api_base = "https://YOUR_RESOURCE.openai.azure.com/"
openai.api_key = "..."  # ⚠️ Hardcoded in many services
# All API calls over HTTPS

3. Azure TTS (Text-to-Speech):

import azure.cognitiveservices.speech as speechsdk

speech_config = speechsdk.SpeechConfig(
    subscription="...",
    region="centralindia"
)
# WebSocket secure (WSS) for streaming

API Communication Encryption¶

External API Calls:

✅ HTTPS for all external APIs
✅ Certificate pinning (recommended, not implemented)
✅ Timeout enforcement
⚠️ No mutual TLS (mTLS) yet

Internal Microservice Communication:

Service A → Gateway (Port 8000) → Service B
   ↓                ↓                  ↓
 HTTPS            HTTPS              HTTPS

Current:

⚠️ HTTP used for internal communication (Docker network)
✅ Network isolation prevents external access
⏳ HTTPS for internal services (planned for production)

Recommended (Production):

Service A → Gateway → Service B
   ↓          ↓          ↓
  TLS 1.3   TLS 1.3   TLS 1.3

WebSocket Encryption¶

Analytics Real-Time Updates:

// Frontend WebSocket connection
const ws = new WebSocket("wss://api.machineavatars.com/ws/analytics");

Security:

✅ WSS (WebSocket Secure) protocol
✅ TLS 1.3 encryption
✅ Authentication token validation
⏳ Connection rate limiting (planned)

🔑 Key Management¶

Current Implementation (Environment Variables)¶

Storage: Docker environment variables, .env files

Services Using Env Vars:

# .env file structure
MONGO_URI=mongodb+srv://...
JWT_SECRET=your_jwt_secret_256_bit
AZURE_OPENAI_KEY=sk-...
AZURE_TTS_KEY=...
GROQ_API_KEY=gsk_...
TOGETHER_API_KEY=...

Benefits:

✅ Not committed to git (.env in .gitignore)
✅ Easy to rotate (update env and restart)
✅ Per environment (dev, staging, prod)

Limitations:

⚠️ Secrets visible in process environment
⚠️ No automatic rotation
⚠️ No audit trail
⚠️ Risk of accidental exposure

Azure Key Vault Integration (PLANNED)¶

Status: Q1 2025 implementation

Architecture:

graph TB
    subgraph "Azure Key Vault"
        KV[Secrets Storage]
    end

    subgraph "Backend Services"
        S1[Auth Service]
        S2[User Service]
        S3[Response Services]
    end

    subgraph "Managed Identity"
        MI[Azure Managed Identity]
    end

    S1 --> MI
    S2 --> MI
    S3 --> MI
    MI --> KV

    style KV fill:#E3F2FD
    style MI fill:#FFF3E0

Implementation Plan:

from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient

# Initialize Key Vault client
key_vault_url = "https://machineavatars-kv.vault.azure.net/"
credential = DefaultAzureCredential()
secret_client = SecretClient(vault_url=key_vault_url, credential=credential)

# Retrieve secrets
def get_secret(secret_name):
    try:
        secret = secret_client.get_secret(secret_name)
        return secret.value
    except Exception as e:
        logger.error(f"Failed to retrieve secret {secret_name}: {e}")
        raise

# Usage in services
MONGO_URI = get_secret("mongo-connection-string")
JWT_SECRET = get_secret("jwt-secret-key")
AZURE_OPENAI_KEY = get_secret("azure-openai-api-key")

Secrets to Migrate:

Secret Name	Current Storage	Target Key Vault Name
MongoDB URI	`.env`	`mongo-connection-string`
JWT Secret	`.env` (weak default)	`jwt-secret-key`
Azure OpenAI Key	Hardcoded!	`azure-openai-api-key`
Azure TTS Key	Hardcoded!	`azure-tts-api-key`
Azure Email Key	Hardcoded!	`azure-email-access-key`
Groq API Key	Hardcoded!	`groq-api-key`
Together AI Key	Hardcoded!	`together-api-key`
Llama API Key	Hardcoded!	`llama-api-key`
reCAPTCHA Secret	`.env`	`recaptcha-secret-key`
Razorpay Key	`.env`	`razorpay-api-key`
Razorpay Secret	`.env`	`razorpay-api-secret`

Benefits:

✅ Centralized secret management
✅ Automatic secret rotation
✅ Access audit logs
✅ RBAC for secret access
✅ Encryption at rest (FIPS 140-2 Level 2)
✅ Managed identities (no credentials in code)

Timeline: Q1 2025 (CRITICAL PRIORITY)

Secret Rotation Policy¶

Current: Manual rotation (when compromised)

Planned:

JWT Secret: 90 days
API Keys: 180 days (or as required by provider)
Database Credentials: 365 days
Emergency Rotation: Immediate on detection of compromise

Rotation Process:

Generate new secret in Key Vault
Update secret reference (no code changes)
Restart affected services
Verify functionality
Revoke old secret after 24-hour grace period

🏢 Customer-Managed Encryption Keys (Enterprise)¶

Availability: Premium Plan only
Status: Planned (Q3 2025)

Bring Your Own Key (BYOK):

Customers can provide their own encryption keys for:

Database encryption (Cosmos DB)
File storage encryption
Backup encryption

Process:

graph LR
    A[Customer] -->|1. Generate Key| B[Customer Key Vault]
    B -->|2. Grant Access| C[MachineAvatars<br/>Managed Identity]
    C -->|3. Use Key| D[Azure Cosmos DB]
    D -->|4. Encrypt Data| E[Encrypted Storage]

    style B fill:#FFE082
    style E fill:#E3F2FD

Requirements:

Customer must have Azure subscription
Customer creates and manages their own Key Vault
Grant wrap and unwrap permissions to MachineAvatars Managed Identity
Customer retains full control over key

Benefits:

✅ Customer controls encryption keys
✅ Can revoke access at any time
✅ Compliance with data sovereignty requirements
✅ Additional layer of security

Limitations:

⚠️ Customer responsible for key management
⚠️ Key loss = permanent data loss
⚠️ Additional complexity and cost

🔐 Encryption Standards & Compliance¶

Algorithms Used¶

Purpose	Algorithm	Key Length	Standard
Data at Rest	AES-256	256-bit	FIPS 197
Data in Transit	TLS 1.3	256-bit (min)	RFC 8446
Password Hashing	None (⚠️)	N/A	NON-COMPLIANT
JWT Signing	HMAC-SHA256	256-bit	RFC 7519

⚠️ CRITICAL ISSUE: Passwords stored in plain text (no hashing)!

Required: bcrypt with 12 rounds minimum

Compliance Mapping¶

GDPR (Article 32 - Security of Processing):

✅ Encryption of personal data at rest
✅ Encryption in transit
⚠️ Password hashing (not implemented)

DPDPA 2023 (Section 8 - Data Security):

✅ Reasonable security practices
✅ Encryption for sensitive data
⚠️ Password protection inadequate

HIPAA (if applicable for on-premise):

✅ Encryption at rest (§ 164.312(a)(2)(iv))
✅ Encryption in transit (§ 164.312(e)(1))
⚠️ Access controls need improvement

PCI DSS (via Razorpay):

✅ No card data stored (handled by Razorpay)
✅ TLS for payment API calls
✅ Razorpay is PCI DSS Level 1 certified

🔧 Implementation Best Practices¶

Encryption Checklist¶

Data at Rest:

MongoDB encryption enabled
Milvus storage encrypted
File uploads encrypted via GridFS
Backups encrypted
Customer-managed keys (Enterprise, planned)

Data in Transit:

HTTPS enforced for frontend-backend
TLS for backend-database
TLS for internal microservices (planned)
WSS for WebSocket connections
Certificate pinning (recommended)

Key Management:

Secrets in environment variables
Azure Key Vault integration (Q1 2025)
Automatic key rotation (Q1 2025)
Audit logs for key access (Q1 2025)

Password Security:

bcrypt hashing (CRITICAL - Q1 2025)
12 rounds minimum
Password migration plan

Security Recommendations¶

Immediate (Q1 2025):

Implement bcrypt for passwords - CRITICAL
Migrate to Azure Key Vault - HIGH
Remove hardcoded secrets - HIGH
Enable TLS for Milvus - MEDIUM

Short-term (Q2 2025): 5. TLS for internal microservices - MEDIUM 6. Certificate pinning - MEDIUM 7. Automatic key rotation - LOW

Long-term (Q3-Q4 2025): 8. Customer-managed keys - Enterprise feature 9. End-to-end chat encryption - Optional feature 10. Hardware Security Modules (HSM) - For key storage

📊 Encryption Performance Impact¶

Benchmarks¶

MongoDB Encryption:

Read Latency: +0-5ms (negligible)
Write Latency: +0-10ms (negligible)
Throughput: No significant impact

TLS 1.3:

Handshake: ~30-50ms (initial connection)
Data Transfer: +0-2% overhead
Resume Session: <10ms (cached)

Azure Key Vault:

Secret Retrieval: 50-200ms (first request)
Caching: <1ms (subsequent requests)
Recommendation: Cache secrets in memory

Security:

Authentication & Authorization - JWT, session management
Secret Management - API key management, migration plan
Network Security - Firewall, VNet isolation

Infrastructure:

Azure Services - Cloud infrastructure
Database Architecture - MongoDB, Milvus setup

Compliance:

Compliance Controls - GDPR, DPDPA, HIPAA requirements

"Encryption is not optional. It's a requirement." 🔐✅

Encryption¶

🎯 Overview¶

🔒 Data at Rest Encryption¶

MongoDB Database Encryption¶

Encryption Details¶

Milvus Vector Database Encryption¶

File Storage Encryption (GridFS)¶

Backup Encryption¶

🌐 Data in Transit Encryption¶

TLS 1.3 Implementation¶

Frontend to Backend¶

Backend to Database¶

Backend to Milvus¶

Backend to Azure Services¶

API Communication Encryption¶

WebSocket Encryption¶

🔑 Key Management¶

Current Implementation (Environment Variables)¶

Azure Key Vault Integration (PLANNED)¶

Secret Rotation Policy¶

🏢 Customer-Managed Encryption Keys (Enterprise)¶

🔐 Encryption Standards & Compliance¶

Algorithms Used¶

Compliance Mapping¶

🔧 Implementation Best Practices¶

Encryption Checklist¶

Security Recommendations¶

📊 Encryption Performance Impact¶

Benchmarks¶

🔗 Related Documentation¶