AI Engineer
Expert in building production-ready LLM applications, from simple chatbots to complex multi-agent systems. Specializes in RAG architectures, vector databases, prompt management, and enterprise AI deployments.
Decision Points
RAG Component Selection
Query Type Assessment:
├── Simple FAQ/Knowledge Lookup
│ ├── Document Count < 1000 → Chroma + text-embedding-3-small
│ └── Document Count > 1000 → Pinecone + text-embedding-3-large
├── Technical/Code Documentation
│ ├── Budget Constrained → bge-large + pgvector
│ └── Performance Critical → voyage-2 + Weaviate
└── Conversational/Multi-turn
├── Memory Required → Agent pattern + context management
└── Stateless → Standard RAG pipeline
Reranking Decision:
├── Precision Critical (legal, medical) → Always use Cohere Rerank
├── Latency < 200ms → Skip reranking, tune retrieval
├── Budget Constrained → Cross-encoder (bge-reranker-large)
└── Default → Cohere Rerank with top-10 → top-3
Database Selection:
├── Existing Postgres → pgvector extension
├── Need Hybrid Search → Weaviate or Qdrant
├── Managed Service → Pinecone
└── Self-hosted/Local → Chroma or Qdrant
Model Routing Strategy
Complexity Assessment:
├── Keywords Only (FAQ) → Claude Haiku
├── Single Document Reference → Claude Sonnet
├── Multi-document Synthesis → Claude Opus
└── Code Generation → Claude Sonnet with tools
Token Budget Check:
├── < 1K tokens → Any model
├── 1K-4K tokens → Sonnet/GPT-4
├── 4K-32K tokens → Claude Opus
└── > 32K tokens → Chunk and summarize first
Agent vs RAG Decision
Task Classification:
├── Static Knowledge Query → Pure RAG
├── Need External APIs → Agent with tools
├── Multi-step Reasoning → Agent with planning
├── Real-time Data Required → Agent with live tools
└── Simple Q&A → RAG with fallback to agent
Failure Modes
Semantic Mismatch Cascade
Symptoms: Good retrieval precision but poor answer relevance, users say "close but not quite right"
Detection Rule: If semantic similarity > 0.8 but user satisfaction < 60%
Root Cause: Query and document embeddings optimized for different semantic spaces
Fix: Switch to domain-specific embedding model or implement query expansion with synonyms
Context Window Overflow
Symptoms: Responses become generic, model ignores specific retrieved context, inconsistent answers
Detection Rule: If context utilization ratio < 30% and response generality score > 0.7
Root Cause: Too many irrelevant chunks diluting relevant information
Fix: Implement stricter relevance threshold (>0.8) and dynamic context selection
Tool Hallucination Loop
Symptoms: Agent makes up API calls, references non-existent functions, infinite retry cycles
Detection Rule: If tool call success rate < 50% or iteration count > max_iterations * 0.8
Root Cause: Model trained on different tool schemas than implementation
Fix: Add tool validation layer and explicit error handling in agent system prompt
Embedding Drift Degradation
Symptoms: Gradual decline in retrieval quality over time, seasonal performance drops
Detection Rule: If monthly average retrieval@5 drops > 10% from baseline
Root Cause: Domain language evolves but embedding model remains static
Fix: Implement embedding model retraining pipeline or switch to adaptive embeddings
Response Latency Creep
Symptoms: P95 latency increases gradually, user complaints about slow responses
Detection Rule: If P95 response time > 2x baseline for 7 consecutive days
Root Cause: Vector index degradation, context size inflation, or model endpoint saturation
Fix: Implement index optimization schedule, context pruning, and multi-model load balancing
Worked Examples
Example: Customer Support Chatbot Implementation
Initial Requirements: "Build a chatbot that can answer questions about our 500-page product documentation"
Step 1: Architecture Decision
- Document count: 500 pages → Use Pinecone for scalability
- Query type: Mixed FAQ + troubleshooting → Hybrid search needed
- Latency requirement: < 3 seconds → Include reranking but optimize
Step 2: Implementation Walkthrough
// Novice approach - would use basic similarity search
const chunks = await vectorDb.query(queryEmbedding, { topK: 5 });
// Expert approach - considers relevance thresholds
const rawChunks = await vectorDb.query(queryEmbedding, {
topK: 20,
threshold: 0.7 // Ensure minimum relevance
});
// Expert adds reranking step novice would skip
const reranked = await reranker.rank(query, rawChunks);
const finalChunks = reranked.slice(0, 3);
// Expert includes fallback handling
if (finalChunks.length === 0) {
return await fallbackToGeneralSupport(query);
}
Step 3: Performance Optimization Discovery
- Initial P95 latency: 4.2 seconds (above requirement)
- Analysis: 60% of time spent in reranking
- Trade-off Decision: Switch from Cohere Rerank to local cross-encoder
- Result: P95 latency → 2.1 seconds, slight quality drop (92% → 89% satisfaction)
- Expert Insight: For support use case, speed > perfect accuracy
Step 4: Failure Scenario Handling
- Discovered 15% of queries were about features not in documentation
- Novice: Would return "I don't know"
- Expert: Added escalation detection and handoff to human agent
Final Architecture: Pinecone + local reranker + agent escalation = 89% automation rate at 2.1s P95
Quality Gates
Not-For Boundaries
Do NOT use this skill for:
Prompt Engineering Tasks → Use prompt-engineer instead
- Optimizing prompt templates and instructions
- A/B testing prompt variations
- Chain-of-thought prompt design
ML Model Training/Fine-tuning → Use ml-engineer instead
- Training custom embedding models
- Fine-tuning LLMs on domain data
- Model architecture research
Data Pipeline Engineering → Use data-pipeline-engineer instead
- ETL processes for training data
- Data validation and cleaning workflows
- Batch processing systems
Infrastructure/DevOps → Use backend-architect instead
- Kubernetes deployment strategies
- Database optimization and sharding
- Load balancer configuration
Analytics and Monitoring Setup → Use chatbot-analytics instead
- Conversation flow analysis
- User behavior tracking
- Performance dashboard creation
Delegate When:
- Task requires deep ML expertise →
ml-engineer
- Focus is on conversation design →
prompt-engineer
- Need infrastructure scaling →
backend-architect
- Want usage analytics →
chatbot-analytics
- Building non-AI features → Relevant specialist skill