Vijaya Karthik
Generative AI Engineer
I build multimodal agentic RAG — AI that answers questions across documents, spreadsheets, audio and video.
Every model runs on open weights, self-hosted on a GPU I run myself — no proprietary API anywhere in the pipeline.
About Me
Building AI That Actually Works
I'm a Generative AI Engineer focused on building AI systems that hold up under real evaluation, not just demos. I care as much about how a system is measured and deployed as how it's designed.
I came to engineering from an unusual direction, and it turned out to be an advantage. Four years preparing for the UPSC civil services examination is four years of synthesizing vast, unstructured source material under evaluation pressure — a discipline that maps directly onto AI engineering, where an answer is only as good as the source it can be traced back to.
My MA in English shows up in the work too: I document systems clearly and report results honestly, including where they fall short. I'd rather ship a measured weakness than an unmeasured claim.
Technical Stack
Skills & Technologies
The stack behind MAGIK — from LLM inference to production infrastructure.
Generative AI & LLMs
Retrieval & Vector Search
Multimodal & Applied ML
Responsible AI & Safety
Backend & APIs
Cloud & Infrastructure
Observability & MLOps
Languages & Tools
Featured Project
Production AI Systems
Multimodal Agentic RAG
Integrated Knowledge AI Assistant (MAGIK)
vjkarthik98/MULTIMODAL-AGENTIC-RAG-INTEGRATED-KNOWLEDGE-AI-ASSISTANTA fully open-source, self-hosted agentic RAG system spanning 7 data modalities for finance-domain Q&A — every model runs on open weights (no proprietary API dependency), deployed on an AWS GPU with enforced tenant isolation, guardrails, and a CI-gated eval harness.
Demo Login
magikaiassistant@gmail.com
Demo@2026
Pre-loaded with sample files & chat history
Key Achievements
- Built a multimodal assistant that answers questions across 7 data types — text, PDF, Word, Excel, images, audio, and video — each handled by its own isolated pipeline, so a problem in one can't break the others.
- Built an intelligent query router that automatically picks the right way to answer each question — searching internal documents, searching the web, or a blend of both — rather than relying on one fixed approach.
- Combined keyword and semantic search with a re-ranking step to surface the most relevant results, tuned for fast, sub-second responses.
- Secured the system for multiple users with proper account authentication (including Google sign-in and multi-factor authentication) and strict data isolation between accounts.
- Built a dedicated safety layer that screens every request for prompt injection, jailbreak attempts, and personal data exposure, tested against a large simulated-attack suite.
- Added full production observability — live dashboards, distributed tracing, and centralized logs — so any request can be traced end-to-end.
- Built an automated testing and evaluation pipeline that blocks any code change from shipping if it makes accuracy or safety worse. Full methodology and results are in the linked System Card.
- Runs 18 open-weight AI models entirely on self-hosted infrastructure, with no dependency on any third-party AI API, and automatic integrity checks that once caught a corrupted model file before it reached production.
- Deployed on AWS with automated CI/CD and a cost-saving design that shuts the GPU down when idle, cutting hosting costs roughly 100x (from ~$1,340/month to ~$12/month) versus running it continuously.
System Card
Model Details
- Version
- v1.0.2 · released 12 Sep 2026
- System
- Retrieval-augmented generation, agentic multi-modal, finance-domain — a compound system, not one trained model
- Core LLM
- Qwen2.5-14B-Instruct (GGUF, Q4_K_M quantized) via llama.cpp
- Models
- 18 open-weight checkpoints · ~42GB on disk · 0 fine-tuned — each pinned to a commit hash, SHA-256 verified on every run
- Retrieval
- BGE embeddings + BGE cross-encoder reranker + Qdrant hybrid search (BM25 + dense) + MMR
- License
- MIT — fully open-weights stack, no proprietary API dependency
- Deployment
- Self-hosted on AWS (NVIDIA L40S 48GB GPU) · Lambda wake gateway, scale-to-zero · Terraform · Docker
Input Modalities · 7
Sample Datasets · real-world test data, 7 modalities
Hosted on Google Drive — each card opens its file directly in a new tab.
Intended Use
Finance-domain document Q&A over uploaded text, PDF, DOCX, XLSX, images, audio, and video (e.g., earnings calls, 10-Ks, financial spreadsheets, chart images). Designed for multi-tenant deployment with strict per-user data isolation.
Out of Scope / Limitations
- Built and evaluated specifically for finance-document Q&A — not a general-purpose open-domain chatbot.
- Retrieval quality is currently under active investigation on one internal benchmark; not yet where it needs to be, and being worked on before the next release. Full detail in the System Card.
- A rare, root-caused hallucination on dense audio transcripts is still open — the model can occasionally mix up two similar numbers from the same document.
- Audio and video answer quality currently trails text, PDF, and spreadsheet inputs, and is the active focus of the next hardening pass.
- Executes one tool call per query (classify → route → execute) — a bounded action, not an open-ended autonomous agent.
- Runs as a single-region demo deployment with no horizontal scaling or failover, a deliberate cost tradeoff (~100x cheaper) for a portfolio project rather than production-scale infrastructure.
Ethical Considerations
- No external inference provider — document and query content never leaves the deployment boundary for a third-party LLM API. This is the system's single largest privacy property.
- Guardrail layer treats all ingested content (documents, audio, web results) as untrusted data, never as instructions — PII detection (Presidio) and toxicity screening (Detoxify) run across ingestion and output.
- No user data crosses tenant boundaries at any storage layer — enforced independently at Qdrant, Redis, MongoDB, and BM25, never delegated upstream to a shared filter.
- Every guardrail violation is written to a persistent audit log, and every cited number is traced back to source text via a finance-fidelity check rather than trusted on the model's word.
Credentials
Advanced Certifications
Enterprise-grade certifications validating expertise in the modern AI stack.
Mathematics — Basics to Advanced for Data Science and GenAI
Udemy
IBM Generative AI Engineering Professional Certificate
IBM · Coursera
MLOps | Machine Learning Operations Specialization
Duke University · Coursera
Responsible AI for Developers: Privacy & Safety
Google Cloud · Coursera
Professional Background
MAGIK AI Assistant (Independent Project) · March 2026 – August 2026
Full-time foundation in ML, deep learning, transformer architectures, and MLOps — three professional certificates, carried straight into production work · June 2025 – March 2026
UPSC Civil Services Examination — four years synthesizing large volumes of unstructured source material under evaluation pressure · June 2021 – May 2025
Bishop Heber College · 2021