Sifu Quest
Client project: Shipmunk AI Lab (Project Owner)
Technical Deep Dive
Built an AI-powered career coaching platform with multi-provider LLM orchestration (Anthropic + OpenRouter), 5 specialized coaching modes, persistent memory system, and enterprise-grade security on Next.js 16 + Supabase.
Client Context
Career-seekers needed personalized AI coaching across DSA, system design, interview prep, job search, and business ideas, with persistent memory and multi-model flexibility.
Execution
Designed a provider abstraction layer supporting Anthropic and OpenRouter with intelligent fallback routing, BYOK encryption (AES-256-GCM), dynamic prompt assembly from user memory files, and SSE-based streaming with provider-specific parsers and optimistic UI updates.
Outcome
Cut LLM time-to-first-token by 60% through prompt caching, parallelized state management, and eager triggering. Implemented GDPR-compliant data architecture with Row Level Security, sliding-window rate limiting (30 req/min), and Sentry observability across Client/Server/Edge runtimes.
Why It Mattered
The highest leverage in applied AI turned out to be fewer token round-trips and running code securely on the learner's own device: a faster first word from the coach, and code sessions where nothing the learner writes has to leave the browser.
“Architected a multi-persona AI coaching engine with real-time SSE streaming and multi-provider routing, cutting LLM time-to-first-token by 60% through prompt caching and parallelized state management.”
Core Stack
Metrics
ttft_reduction
60%
cache_hit_rate
90%
coaching_modes
5
rate_limit
30 req/min