Enterprise AI assistant

Why

The organization needed one credible intelligent layer instead of a dozen one-off experiments that could not pass security or scale review.

Pain points

• Experiments lived in silos — different prompts, no shared evals, no consistent security story.
• Legal and compliance needed clear provenance, retention, and access patterns for model-backed answers.
• Teams wanted to move fast without reinventing retrieval, tool execution, and observability for every use case.

Overview

A governed assistant layer for internal teams: users ask in natural language, the system retrieves from approved sources, calls tools when appropriate, and leaves an audit trail suitable for compliance review. Built for multiple tenants with strict data isolation.

Architecture

A synchronous API path handles chat: authenticate → resolve tenant → retrieve (with ACLs) → optional tool calls → model with grounded context → structured logging. Async workers handle ingestion, embedding refresh, and batch eval jobs. Tenant identity flows through every hop so data never crosses boundaries by accident.

Technical deep dive

Services in TypeScript and Python; PostgreSQL for tenancy and workflow state; vector store for embeddings with versioned ingestion pipelines. Orchestration on Kubernetes with separate paths for synchronous chat and async batch jobs. Evaluation pipelines compared model versions against golden sets and production samples — not just offline demos.

What I did

• Owned architecture for retrieval, tool execution, and tenancy boundaries across services.
• Partnered with security and compliance on logging, retention, and access patterns.
• Led incremental rollout: pilot teams first, then hardening before wider adoption.
• Mentored engineers on LLM ops: eval harnesses, latency budgets, and cost visibility.

Outcomes

• Assistant adopted by multiple business units with shared governance and monitoring.
• Grounded answers with citation-style provenance reduced “hallucination” incidents in pilot reviews.
• Unified observability: traces from user message through retrieval, model call, and tool steps.

Designed for high availability and clear SLOs; tenant-isolated data paths and audit logs as first-class concerns.