What is AI voice cloning, and how does it actually work?

October 22, 2026 · 6 min read · By Resona Editorial Team · Resona Blog

Voice cloning is the technique of teaching a speech model to talk in one specific voice. Give it a short recording of a person, and it can then read any text in that voice — including words the original speaker never said. This article explains what actually happens under the hood, in plain language, and where the limits are.

1. What the model learns from your sample

Modern systems use few-shot voice cloning: from a sample as short as 10–30 seconds, the model extracts a compact representation of the voice — pitch range, resonance, articulation habits, speaking rate, and accent. It does not store your recording, and it does not need hours of studio data. What you get back is a reusable voice model: type new text, and it speaks in the sampled voice.

Two things travel together in that representation:

Timbre — the "colour" of the voice that makes it recognizable. This is what most people mean when they say a clone sounds like someone.
Prosody — rhythm, stress and intonation. A longer sample (a minute or more) captures more of it, because the model needs to hear how the person phrases things, not just how they sound.

2. How text becomes speech

Once the voice model exists, generating speech is a three-step pipeline:

Text analysis — the input text is converted into phonemes (the sound units of a language), with punctuation and structure read as cues for pauses and emphasis.
Acoustic modelling — the phonemes plus your voice representation become an intermediate acoustic representation: roughly, a spectrogram describing which frequencies are present at each moment.
Vocoding — the acoustic representation is rendered into the final audio waveform you download.

Emotion and style controls sit on top of this pipeline. A tag like "thoughtful" or "excited" biases the prosody of the output without changing the underlying voice — which is why the same clone can read a calm narration and an energetic ad.

3. Zero-shot libraries: cloning without a sample

There is a second way to get a voice: pick one from a library. Large public libraries — Resona's, for example, holds more than 200,000 voices — are built from consented, licensed recordings, and any of them can be used immediately without uploading a sample of your own. That is called zero-shot usage: the voice already exists as a model, and your text simply drives it.

For most projects the two approaches complement each other: clone a specific person when identity matters (your own voice, a narrator you have rights to), and use a library voice when you just need a good fit for the role.

4. What cloning cannot do (yet)

Honest limits, because marketing pages rarely state them:

Mannerisms are not cloned. Filler words ("um", "you know"), habitual pauses and verbal tics are not in the text you type, so no TTS system reproduces them. You write them into the script, or clone from audio where they occur naturally.
Quality varies by language. Major languages are close to indistinguishable from the original; rarer languages can still sound synthetic even with a good sample.
Singing is out of scope. These models are built for speech. Pitch-accurate singing is a different problem.
Emotion is steerable, not automatic. Without reference audio or tags, output tends toward neutral delivery.

5. What decides whether your clone sounds like you

In order of impact: sample cleanliness (room noise hurts more than mic quality), sample content (conversational speech clones better than read-aloud text), sample length for prosody, and the model tier behind the scenes. Our sample recording guide walks through each of these with a checklist.

A test you can run in five minutes

Clone from a casual 30-second recording, then generate the same sentence twice — once in a news-anchor style, once in an excited style. If both versions sound like the same person talking, the clone captured the voice correctly. If only one does, your sample probably needs more conversational variety.

Ready to try it? Open the Resona console, upload a short sample, and compare the result against the original. Cloning costs 100 credits (about $2), and credits never expire — there is no subscription, and you pay for what you generate rather than for the right to try.

Rates are on the pricing page: cloning a voice is 100 credits (about $2) once, speech is billed by the byte, and credits never expire. Start from the Resona console when you are ready.

New to this topic? Read the full Voice Cloning Guide.