Design an Agent Evaluation Harness
Helps you design an agent evaluation harness quickly, while still surfacing the open questions a real version of this work would need answered.
When to use this
Use before committing time to design an agent evaluation harness by hand, to get a first structured pass to react to.
Prompt template
Act as an AI agent architect. Design for reliability under partial failure: assume tools can time out, return malformed data, or be unavailable, and design accordingly. State explicit assumptions about the underlying model's capabilities rather than assuming a specific one.
Task: Design an agent evaluation harness.
Context to use:
- What the agent (or agent system) is trying to achieve: {{system_goal}}
- Available tools or integrations: {{tools_or_integrations}}
- Constraints and risk tolerance: {{constraints}}
Response format:
1. Brief recommendation or approach
2. Main deliverable, organized into clear sections
3. Assumptions and unresolved questions
4. Validation or quality checklistCustomize
What the agent (or agent system) is trying to achieve is required
Preview
Act as an AI agent architect. Design for reliability under partial failure: assume tools can time out, return malformed data, or be unavailable, and design accordingly. State explicit assumptions about the underlying model's capabilities rather than assuming a specific one.
Task: Design an agent evaluation harness.
Context to use:
- What the agent (or agent system) is trying to achieve: {{system_goal}} missing
- Available tools or integrations:
- Constraints and risk tolerance:
Response format:
1. Brief recommendation or approach
2. Main deliverable, organized into clear sections
3. Assumptions and unresolved questions
4. Validation or quality checklistUsage instructions
Provide as much real context as you have; leave a field blank only if it genuinely doesn't apply, and check the assumptions list afterward.
Example output
A structured write-up covering the approach, the concrete steps or artifacts needed, open questions, and a checklist to confirm it's ready to use.
Limitations
Treat the output as a structured starting point; it does not replace review from someone with direct context on the system.
Rate this prompt
Not yet rated