Build an evaluation process for AI agents using success, reliability, cost, and latency measures. Define scoring criteria, repeatable environments, failure analysis, and regression tests that check for recurring problems to support model selection and system improvements.
- Building an initial agent evaluation process
- Comparing models on actual work tasks
- Investigating a drop in evaluation scores
bojieli/ai-agent-book☆ 54.8KUpdated Oct 9, 2026
Design a process for AI agents to learn from execution records. Evaluate outcomes, actions, and quality, choose whether to update knowledge, instructions, code, or model parameters, and define independent validation, release checks, and rollback procedures.
- Turning repeated agent failures into reusable guidance
- Designing evidence-based reviews of execution records
- Converting browser actions into validated workflows
bojieli/ai-agent-book☆ 54.8KUpdated Oct 9, 2026
Design a running status summary that keeps an agent aware of tasks, tool call counts, time, errors, and its environment. The skill covers updating that summary and balancing cache reuse against the buildup of outdated information.
- Addressing repeated tool calls and retry loops
- Keeping TODO progress and original goals visible
- Choosing how to update runtime status messages
bojieli/ai-agent-book☆ 54.8KUpdated Oct 9, 2026
Review and rewrite Chinese product announcements, articles, emails, and README text using eleven style checks. Identify repetitive phrasing, overloaded sentences, vague openings, and decorative language, then revise them into clearer statements suited to each type of writing.
- Revising formulaic Chinese product announcements
- Reviewing repetitive phrasing in Chinese articles
- Shortening overloaded sentences in Chinese emails
bojieli/ai-agent-book☆ 54.8KUpdated Oct 9, 2026
Design agents that respond to incoming emails, external notifications, and timers. Plan how they handle interruptions, prioritize new requests, track concurrent tasks, and deliver results while longer operations continue in the background.
- Connecting agents to incoming emails and external notifications
- Scheduling reports and recurring checks
- Handling interruptions and changes during long tasks
bojieli/ai-agent-book☆ 54.8KUpdated Oct 9, 2026
Turn agent failure records into training examples by locating the first incorrect decision and pairing preferred actions with rejected ones. Assess whether training is appropriate, then check both targeted improvements and preservation of correct behavior.
- Diagnosing the first incorrect decision in an agent failure
- Building preference pairs from failure records
- Choosing between system fixes and model training
bojieli/ai-agent-book☆ 54.8KUpdated Oct 9, 2026