Why voice is harder than chat
A voice agent listens to a caller, works out what they want and replies in synthesized speech, in real time. Each of those steps can go wrong in ways text never does. The caller is in a car, on a poor line, with a child shouting nearby. They interrupt. They spell a name. They pause mid-sentence and the system talks over them.
Callers are also less patient than people typing. A second of silence feels long. A misunderstanding that would be a shrug in chat becomes a reason to hang up. None of this rules voice out. It means the scope should be narrow at first, the conversation carefully written and the route to a person never more than a sentence away.
Choose which calls it takes
Listen to recordings or read call notes for a few weeks and group calls by purpose. For each group, ask whether the call follows a predictable path and whether the information needed is in a system.
| Call type | Example | Fit for a voice agent |
|---|---|---|
| Simple information | Opening hours, location, whether a service is offered | Strong |
| Booking and rescheduling | Make, move or cancel an appointment | Strong, with a connected calendar |
| Status enquiries | Is my order ready, has my payment arrived | Good, after identity is verified |
| Call routing | Finding out what the caller needs and connecting them | Good, and a gentle first use |
| Message taking out of hours | Capture who called and why, promise a call back | Good |
| Complaints and distress | Something has gone wrong and the caller is upset | Transfer promptly |
| Complex or sensitive matters | Medical concerns, financial hardship, legal questions | A person, with the agent only routing |
Write the conversation for the ear
- Short turns. One idea per sentence, one question at a time. Lists of more than three options are forgotten before they finish.
- Fast first response. The delay between the caller finishing and the agent starting is the quality people notice first. Ask whoever builds it to measure this on real calls.
- Interruptions allowed. A caller who speaks while the agent is talking should be heard, and the agent should stop.
- Confirm what matters. Read back dates, times, amounts and names before acting on them. Spell names and reference codes back letter by letter where needed.
- Recover gracefully. After one misunderstanding, rephrase the question. After two, offer a person. Never loop.
- Silence and background noise. Decide how long to wait, how to prompt and when to end a call politely.
- Languages and accents. Name the languages and dialects callers use. Speech recognition quality differs between them, so each needs its own test.
Numbers, addresses, email addresses and names are the hardest things to capture by voice. Where one is essential, consider confirming it by text message during the call.
Design the handoff before anything else
The handoff decides whether callers trust the system. Design it first, and treat every other feature as optional by comparison.
- 01
Define the triggers
The caller asks for a person in any words. Two failed attempts to understand. A topic on the transfer list. Signs of distress or anger. A request the agent is not permitted to handle.
- 02
Choose the kind of transfer
A warm transfer passes a summary to the person before or as they pick up. A cold transfer simply connects the call. Warm is better whenever your phone system allows it.
- 03
Pass the context
Who is calling, whether they were verified, what they asked, what was already done. It should appear on the agent's screen as the call arrives.
- 04
Route to the right person
Use what the agent has learned to pick the queue or individual, with a fallback when that queue is busy.
- 05
Plan for nobody being available
Offer a call back at a stated time or take a message. Say honestly that no one is free. Do not leave the caller on hold with no end.
- 06
Tell the caller what is happening
A sentence before transferring, such as I am connecting you to a colleague and passing on what you told me, keeps the caller on the line.
Disclosure, recording, consent and identity
- 01
Does the agent say it is automated?
What to settleIt should, at the start of the call. Rules on disclosure exist in some jurisdictions, and callers react badly to discovering it later.
- 02
Are calls recorded or transcribed?
What to settleRecording and consent laws differ by country and sometimes by region. Have the opening wording and retention period reviewed by a legal adviser.
- 03
How is a caller verified?
What to settleThe calling number alone is weak evidence. Use details matched against the record, a one-time code by text, or transfer to a person for anything sensitive.
- 04
What may be said aloud?
What to settleOthers may be listening at the caller's end. Avoid reading out full personal or financial details.
- 05
Where does the audio go?
What to settleSpeech services process recordings on their providers' systems. Check the terms for retention, location and use of data, as for any other AI service.
- 06
Are outbound calls planned?
What to settleAutomated outbound calling is restricted in many places. Take legal advice before any system places calls.
A worked example
A voice agent for a group of dental clinics
This example is invented to show a narrow first scope. A group of dental clinics misses many calls during treatment hours and after closing.
- CallsA fortnight of call notes shows that most calls are to book, move or cancel an appointment, or to ask about hours and location. Those four become the scope. Pain and emergencies are on the transfer list.
- ConversationThe agent opens by saying it is the clinic's automated assistant. It asks one question at a time and reads back the date and time before confirming a booking.
- IdentityTo move or cancel, the caller gives a name and date of birth, matched against the booking. Nothing clinical is ever read out.
- HandoffAny mention of pain, bleeding or an emergency triggers an immediate transfer to the front desk, with a one-line summary on screen. Out of hours, the agent gives the emergency number and takes a message.
- TestingThirty staff and volunteers call from mobiles, landlines and cars, in two languages. Names are the weak point, so bookings are confirmed by text message with the name as captured.
The clinics stop missing booking calls, and reception handles the calls that need a person. The text confirmation catches misheard names before they reach the diary.
Test on real lines and measure after launch
Worth measuring
- Calls completed without transfer, by call type
- Transfers that arrived with context
- Callers who hung up, and at which point
- Time before the agent's first response in each turn
- Errors in captured details, found by sampling recordings
Worth avoiding
- Testing only in a quiet office on a headset
- Judging by the share of calls kept from staff
- No review of recordings after launch
- Launching every call type at once
- Removing the option to reach a person
Begin with limited hours or one location, listen to recordings daily in the first weeks and widen only when the figures hold. Our guide on AI workflow approval and human review design covers how to decide which actions need a person's confirmation.
Voice agent planning checklist
AI voice agent planning checklist
0 of 11 checkedWhat to do next
Count your calls by purpose for a fortnight, including the ones that were missed. If a few simple purposes account for most of them, a voice agent has a clear job. If calls are mostly long and varied, start with routing and message taking only.
Our AI agent solutions catalogue includes voice and reception agents by industry. Tell us what your callers ask for and how your phone system is set up, and we will say what a sensible first scope would be.
Common questions
Do callers accept talking to an AI on the phone?
Many do when it answers at once, understands them and completes a simple task, and when a person is available on request. Acceptance falls quickly if they are misunderstood twice or cannot reach anyone. It varies by audience and by the nature of the call, which is why a limited start with close listening is the sensible approach.
How does the agent transfer a call to a person?
Through your phone system. Depending on what that system supports, the agent can connect the caller directly, or first pass a short summary to the person who will answer. The summary can also appear on the person's screen. Ask early what your telephony provider allows, because it shapes the whole handoff design.
Can a voice agent understand different accents and languages?
Speech recognition works well for many languages and accents and less well for others, and performance drops with noise and poor lines. Name the languages and dialects your callers use and test each with real speakers on real phones. Where recognition is weak for a group of callers, route them to a person early.
Is it legal to record and transcribe calls handled by AI?
That depends on where you and your callers are. Many jurisdictions require callers to be told, and some require consent. Rules may also cover how long recordings are kept and automated calling. Have your opening message, retention period and any outbound use reviewed by a legal adviser for each market you serve.
Should we replace our phone menu with a voice agent?
A voice agent that asks what the caller needs and routes accordingly is often a better experience than a long keypad menu, and it is a low-risk first step. Keep a simple fallback, such as pressing zero for a person, for callers the agent cannot understand, and review routing accuracy before adding tasks beyond routing.
