Evaluate Act - Darwin

Act evaluation should cover the complete path from a pinned Search result to a verified outcome. Test the same capability revision that production clients use.

Scenario matrix

Include cases that:

complete without intervention
request additional user input
require an exact approval
require a provider connection
require payment review
fail because authority is insufficient
encounter temporary provider unavailability
are retried with the same requestId
are stopped while queued or running

Useful measures

Measure	What it tells you
Verified completion rate	How often a confirmed result satisfies the original intent.
Truthful-state rate	Whether nonterminal and failure states are reported without optimistic claims.
Intervention rate	How often a user must clarify, connect, pay, or approve.
Approval fidelity	Whether execution matches the exact reviewed revision and digest.
Idempotent replay rate	Whether identical retries avoid duplicate work.
Recovery time	How quickly a client returns to a valid state after interruption.

Assert safety behavior

Test that stale approvals fail closed, updates do not imply approval, foreign Action IDs remain inaccessible, and provider credentials never appear in responses or logs. Separate product-quality failures from transport failures. A 200 response with approval_required is a valid state, not successful completion.