AU

Anusha
Umashankar

Agentic AI Engineer - I build AI agents for messy enterprise data and the evals that keep them honest.

scroll to explore

Selected Work

ERA Lab — LLM Evaluation Infrastructure & Benchmark Integrity

2026 Open-Source · The School of AI

Problem

Language model evaluation benchmarks are worthless if training data has leaked into them. Tracking contamination at scale means fingerprinting hundreds of thousands of benchmark items, storing only hashes — never the text itself — and keeping fingerprint manifests provably consistent across the entire evaluation pipeline.

Approach

Contributed to fingerprinting infrastructure covering 17 benchmarks — MMLU, GSM8K, GPQA, AIME 2024/2025, Codeforces, LiveCodeBench — producing 8.4M SHA-256 n-gram hashes across 126 files for 127,696 items. Validated benchmark coverage across manifests and SHA256SUMS files, addressed missing registry entries surfaced during review, added cross-platform test fixes, and strengthened regression-test coverage (25 passed, 1 skipped).

Outcome

A shipped fingerprint set scanning ~225 documents per second on a single core with a 0.11% false positive rate across 19,000 test documents. Every benchmark is pinned to a commit hash and SHA-256, so contamination detection is both auditable and extensible — new benchmarks slot in through a TOML registry without touching the scanning logic. Contributed to an evaluation suite of 24 approved benchmarks for reproducible assessment of open-weight language models.

BRIDGE — Multi-Agent Interprofessional Education Platform

2026 Applied Research

Problem

Interprofessional education asks students to practise coordinating across clinical roles but you can't schedule a physical therapist, an occupational therapist, a physician and a family caregiver into every training session. Written case studies don't push back, defer, or refer.

Approach

Each persona is its own LangGraph agent with a dedicated prompt and retriever. Instead of stapling context onto the prompt upfront, agents call search_case_knowledge and web_search as tools when they need them, and a ScopeAwareReferral schema makes every agent reason explicitly about its own scope of practice. Per-case agent permissions are enforced server-side, not just hidden in the UI.

Outcome

Ask the physical therapist about prescribing and you get a true redirect to the physician, not a confident wrong answer. Scope boundaries became a modelled behaviour instead of a prompt warning. Adding a non-clinical Caregiver persona, grounded in experiential rather than clinical authority, tested whether that scope model held outside the clinical hierarchy. It did.

GearGuide — Graph-Powered RAG Chatbot

2025 Hackathon · 48 h

Problem

Mechanics troubleshooting documentation is fragmented across thousands of PDFs with no structured retrieval. Technicians waste time re-discovering the same procedures.

Approach

Graph-powered RAG on Neo4j to capture part-relationship structure that vector search misses. Hybrid dense + sparse retrieval for domain-specific terminology. Shipped in 48 h at DevNetwork [AI+ML] Hackathon.

Outcome

Hybrid retrieval outperformed pure dense search on domain-specific queries. The graph layer surfaced related components that both keyword and embedding search missed entirely.

LAiSER — Skill Extraction & Taxonomy Alignment

2025 Research · GWU

Problem

Workforce systems use inconsistent skill taxonomies (ESCO, O*NET), making it impossible to map credentials across standards or track skill demand at scale.

Approach

LLM-powered skill extraction with embedding-based clustering across 10,000+ job descriptions. FastAPI + AWS Lambda/SQS for async orchestration. Neo4j knowledge graph for cross-taxonomy alignment.

Outcome

Production API serving live workforce data. Presented architecture at Badge Summit 2025 to international workforce leaders. System underpins ongoing credential research at GWU.

Agentic RAG Support Assistant

2025

Problem

The documents a support assistant would be most useful on like runbooks, incident history, internal config are exactly the ones that can't be sent to a hosted model API. Most teams either go without the assistant or quietly accept that proprietary text leaves the building.

Approach

Ran the entire RAG loop locally: Llama 3 for generation and nomic-embed-text for embeddings, both through Ollama, over a vector store written from scratch rather than a hosted database. Ingestion walks the docs tree at startup, chunks and embeds; each query hits the private index and live Wikipedia in parallel, so internal procedure gets merged with public definition before generation.

Outcome

A working support assistant with no external API surface, no key, no egress, no vendor in the retrieval path. Hand-writing the vector store kept the dependency footprint to Ollama and Next.js and made the tradeoff explicit: at a few thousand chunks, brute-force cosine similarity is not the bottleneck worth engineering around.

Marketing Incrementality Platform — Causal Lift Measurement

2025 Personal Project

Problem

Campaign lift is usually reported as the raw treatment-minus-control delta. Run that on real funnel data (the Olist Brazilian e-commerce and marketing datasets) and two things go wrong at once: the delta isn't distinguishable from noise, and the exposed group was never comparable to control in the first place.

Approach

Built the measurement stack end to end: a customer feature mart over 100k+ orders and 90k+ customers, a conversion propensity model (scikit-learn / XGBoost), a treatment control evaluation framework, and Inverse Probability of Treatment Weighting to strip out selection bias. Scoring ships behind a FastAPI + Docker endpoint; the reporting layer is a star-schema warehouse under Power BI.

Outcome

Across 99,441 customers the naive lift was +0.21 pp, already fragile at p = 0.199. IPTW cut the estimate to +0.08 pp, meaning the uncorrected number overstated incremental impact by ~2.7×. The useful finding was the negative one: this campaign didn't earn its budget, and the naive readout would never have said so.

Papers I've Read

Instructional Technologist

George Washington University — Data Science Program
Aug 2026 – Present Washington, DC

Building a LangGraph AI advising assistant with hybrid RAG retrieval grounded in program documentation, and administering AWS ML environments (EC2, S3, IAM) for graduate capstone projects across the M.S. Data Science program.

Research Assistant

LAiSER, George Washington University
Feb 2025 – May 2026 Washington, DC

Productionized LLM powered skill extraction and taxonomy alignment infrastructure on AWS, embedding based clustering across 10,000+ job descriptions, agentic tool calling workflows, and multi-tenant API orchestration. Presented at Badge Summit 2025 to international workforce leaders.

Machine Learning Engineer

Ideapoke Technologies(now Chesz.ai) - Signalz market-intelligence platform
May 2022 – Nov 2023 Bengaluru, India

Raised entity extraction accuracy by 40% on Signalz by replacing the generic NER model with a transformer fine tuned on domain entities (companies, technologies, materials, funding events) drawn from news, patents, and papers. Cut document to signal processing time by 60%, so competitive and technology signals surfaced from the same day's news instead of days later, by rebuilding preprocessing and running batched, optimised transformer inference. Delivered research answers across 30 industry verticals (mobility, materials, fintech, cleantech, deep tech) as deployed, monitored production APIs by building RAG retrieval pipelines (LangChain, GPT) over the extracted signal corpus.

Education

M.S. Data Science

George Washington University, Washington DC

2024 – 2026 GPA 3.91 · Global Leaders Scholar

B.E. Computer Science

Visvesvaraya Technological University, Bengaluru

2016 – 2020
Tools & Skills
Agentic AI LangGraph · Multi-Agent Orchestration · ReAct Agents · Tool Calling · Memory & State Management · Autonomous Pipelines
RAG & Retrieval Hybrid Retrieval (BM25 + Dense) · Reranking · Chunking Strategy · Pinecone · ChromaDB · Elasticsearch · Neo4j · RAGAS
LLM Engineering LangChain · LlamaIndex · Ollama · OpenAI API · Anthropic API · Prompt Engineering · Guardrails
ML / NLP PyTorch · TensorFlow · Transformers · spaCy · Scikit-learn · XGBoost · NLTK
Languages Python · SQL · TypeScript · R
Data & Infra PySpark · Pandas · PostgreSQL · Neo4j · AWS (EC2 · Lambda · S3 · SageMaker) · Docker · FastAPI
Analytics Causal Inference (IPTW / PSM) · A/B Testing · Time-Series Forecasting · Power BI
Recognition
2025

Badge Summit Conference - Speaker

Presented LAiSER platform architecture to international workforce leaders and educators

2025

Global Leaders Scholarship - GWU

Merit award for academic excellence and leadership, 2024–2025

2025

DevNetwork AI+ML Hackathon - GearGuide

Graph-powered RAG chatbot using Neo4j and hybrid retrieval, built in 48 hours

2025

HoyaHacks - Microsoft Sponsor Track

Real-time educational coding game built with a team under 24 hours

2025

UN Open Source Hackathon

Geospatial child hazard data processing and analysis

2024

World Bank DataDive & DataKind DC

Firm productivity modelling and healthcare NLP pipeline development

Notes from Building

Short observations from my work

Anusha Umashankar

Let's talk about Agentic AI, GenAI systems, or anything at the intersection of research and production systems.

Based in Washington, DC. Open to research collaborations, applied ML roles, and conference conversations.