Vijaya Karthik
Generative AI Engineer
Building production-ready LLM pipelines and RAG systems that bridge complex AI architectures with real-world, responsible applications.
About Me
Building AI That Actually Works
I'm a highly disciplined Generative AI Engineer with a robust foundation in machine learning operations, responsible AI frameworks, and production LLM systems.
I combine rigorous algorithmic problem-solving with specialized, enterprise-grade training to build and deploy scalable LLM applications. My background in analytical research — including multi-year preparation for one of India's most competitive examinations — gives me an edge in synthesizing complex information into production-ready solutions.
From architecting multimodal RAG pipelines to deploying containerized AI APIs, I bridge complex technical architectures with ethically grounded, real-world impact.
Generative AI
RAG pipelines & agentic systems
MLOps
Docker, MLflow, GitHub Actions & AWS deployment
Responsible AI
Ethical frameworks and evaluation pipelines
Technical Stack
Skills & Technologies
A curated set of tools and frameworks I use to build production AI systems.
Programming
Generative AI & ML
Cloud & MLOps
Core Competencies
Developer Tools
Featured Project
Production AI Systems
Multimodal Agentic RAG
Integrated Knowledge AI Assistant (MAGIK)
A fully open-source, self-hosted agentic RAG system spanning 7 data modalities for finance-domain Q&A — every model runs on open weights (no proprietary API dependency), deployed on an AWS GPU with enforced tenant isolation, guardrails, and a CI-gated eval harness.
Demo Video
Add /public/demo.mp4 to display
Key Achievements
- Architected a per-modality-isolated ingestion pipeline (7 modalities: text, PDF, DOCX, XLSX, image, audio, video), each with independent ingest / chunk / embed / BM25 layers so a bug in one modality can't break another.
- Built an agentic query router that classifies each incoming query (RAG / web search / hybrid / direct / finance-specific) and dispatches to the correct tool via a typed ToolCall / ToolResult contract — no raw dicts cross module boundaries.
- Implemented hybrid retrieval: BM25 + dense vector search (Qdrant) fused, re-ranked with a cross-encoder, then diversified with MMR before generation.
- Enforced multi-tenant data isolation across all four storage layers (Qdrant, Redis, MongoDB, BM25 indices) with per-user JWT auth, Argon2 password hashing, Google OAuth (PKCE), and TOTP MFA.
- Built a consolidated guardrail layer covering prompt-injection and PII detection across all 28 modality × pipeline-layer surfaces, plus output-side response filtering before any answer reaches the client.
- Instrumented the full request path with Prometheus metrics, OpenTelemetry tracing, and structured JSON logging for production observability.
- Built a CI-gated evaluation harness (retrieval + generation quality) that blocks merges on regression — not a one-off benchmark script.
- Runs 10 resident ML models (LLM, embedder, reranker, vision, ASR, diarizer, OCR, NER) concurrently on a single A10G 24GB GPU through careful device / memory management.
Model Card
Model Card
vijaya-karthik/multimodal-agentic-rag
Model Details
- System
- Retrieval-augmented generation, agentic multi-modal, finance-domain
- Core LLM
- Mistral-7B-Instruct (GGUF, Q4_K_M quantized) via llama.cpp
- Retrieval
- BGE-large-en-v1.5 embeddings + BGE-reranker-large + Qdrant + BM25
- License
- Fully open-source / open-weights stack — no proprietary API dependency
- Deployment
- Self-hosted on AWS (A10G 24GB GPU) · llama.cpp · Docker
- Repo
vijaya-karthik/multimodal-agentic-rag
Input Modalities · 7
Intended Use
Finance-domain document Q&A over uploaded text, PDF, DOCX, XLSX, images, audio, and video (e.g., earnings calls, 10-Ks, financial spreadsheets, chart images). Designed for multi-tenant deployment with strict per-user data isolation.
Out of Scope / Limitations
- Not a general-purpose open-domain chatbot — tuned and evaluated specifically for finance-document retrieval and numeric fidelity.
- Audio and video pipelines currently perform below the text/PDF/XLSX baseline (see Evaluation Results) and are actively being hardened.
- Single-dispatch agent (classify → route → execute) — not yet an iterative multi-step tool-chaining loop.
Ethical Considerations
- Guardrail layer treats all ingested content (documents, audio, web results) as untrusted data, never as instructions — mitigates prompt injection from uploaded files.
- No user data crosses tenant boundaries at any storage layer (enforced, not just assumed, at Qdrant / Redis / MongoDB / BM25).
- Every guardrail violation is written to an audit log.
Evaluation Results
Real numbers from the CI-gated eval harness and manual per-modality audits — reported as-is, including where audio and video still trail.
Retrieval & Generation Quality · CI-gated thresholds
| Metric | Threshold |
|---|---|
| Recall@5 | ≥ 0.418 (gate) |
| MRR | ≥ 0.772 |
| nDCG@10 | ≥ 0.575 |
| Faithfulness | ≥ 0.25 |
| Finance numeric fidelity | ≥ 0.95 |
| Routing accuracy | ≥ 0.917 |
Faithfulness uses a harness-specific (Ragas-style) scale, not a 0–1 accuracy percentage.
Per-Modality QA Benchmarks · manual audits /100
| Modality | Score | Status |
|---|---|---|
| XLSX | 92.0 | Pass |
| Image | ~95 | Pass |
| 85.8 | Pass | |
| Audio | 78.8 | Partial — upstream pipeline scores 97/100, gap is in shared query-answering layer |
| Video | 71.75 | In progress — upstream ingestion scores 100/100 |
Credentials
Advanced Certifications
Enterprise-grade certifications validating expertise in the modern AI stack.
IBM Generative AI Engineering Professional Certificate
IBM · Coursera
MLOps | Machine Learning Operations Specialization
Duke University · Coursera
Responsible AI for Developers: Privacy & Safety
Google Cloud · Coursera
Professional Background
Master of Arts in English
Bishop Heber College · 2021
UPSC Examination Preparation
Advanced research & analytical synthesis · June 2021 – May 2025
Independent Generative AI Engineering
Full-time self-directed mastery · June 2025 – Present