Icana Bench
We test AI voice agents on regulated customer calls. We score them the way a compliance team and a call coach would score a person. Did the agent follow the rules? Did it say only what is true? How well did it handle the caller? How long did the caller wait? There are three kinds of call, each scored on its own. All three belong to one made-up brand, Pulse Fitness.
Overview
GPT Realtime 2.1, no reasoning leads Icana Bench 0.9 with 56.7, the average of three domains. It passes every must-pass compliance check on 30% of its calls, averaged over the domains. On call skills, no agent reaches the typical human agent in any of the three domains. The closest is Gemini 3.8 Live Extended Thinking, medium in Finance: 61.5 against 62.8.
| Agent | Icana Bench 0.9 | Finance measured | Partners preview | Coaching preview |
|---|---|---|---|---|
| GPT Realtime 2.1, no reasoning | 56.7 | 60.6 | 57.9 | 51.6 |
| Gemini 3.8 Live | 55.2 | 58.8 | 54.1 | 52.7 |
| GPT Realtime 2.1, reasoning medium | 55.0 | 62.5 | 52.7 | 49.7 |
| Gemini 3.8 Live Extended Thinking, medium | 54.2 | 55.7 | 56.1 | 50.9 |
| GPT Realtime 2.1 Mini, no reasoning | 45.5 | 49.0 | 46.9 | 40.6 |