ePrints

What determines the success of AI voice-cloned speech? Prosodic and acoustic evidence on three TTS systems

Lookup NU author(s): Dr Kai AlterORCiD

Downloads

• Published version [.pdf]

Licence

This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).

Abstract

© 2026 the author(s), published by De Gruyter, Berlin/Boston.Artificial intelligence (AI) voice cloning systems have advanced rapidly, enabling applications in education and assistive technologies. Yet listeners’ perceptual ratings of naturalness and similarity remain inconsistent: some systems approach human-like quality, while others sound noticeably artificial. Here we present a comprehensive prosodic and computational analysis of voice-cloned speech across three voice-cloning systems (ElevenLabs, StyleTTS-V2, XTTS-V2), building on the listener judgments of these stimuli reported in Bakkouche et al. (Finding the human voice in AI: Insights on the perception of AI-voice clones from naturalness and similarity ratings. In Proceedings of Interspeech 2025, 2190–2194. Rotterdam, The Netherlands: ISCA. https://www.isca-archive.org/interspeech_2025/ bakkouche25_interspeech.pdf (accessed 14 October 2025)). We analysed pitch, amplitude, speech rate, rhythm, intonation, and speaker-embedding similarity. Overall, ElevenLabs showed the closest correspondence to human speech across several prosodic and speaker-identity measures, although system differences were not uniform across all dimensions. The clearest acoustic differences between models were observed in speech rate, vowel-based rhythm measures, local pitch-control measures, and speaker-embedding similarity. These acoustic findings are consistent with listeners’ perceptual judgments of naturalness and suggest that prosodic timing, rhythm, and fine-grained pitch control are potential correlates of perceived naturalness, and that improvement of these features can contribute to the development of more natural-sounding synthesised speech.

Publication metadata

Author(s): Bakkouche L, Luo X, Lau E, McGhee C, Cooper S, Post B, Alter K, Schwarz J

Publication type: Article

Publication status: Published

Journal: Phonetica

Year: 2026

Pages: epub ahead of print

Online publication date: 10/06/2026

Acceptance date: 13/05/2026

Date deposited: 29/06/2026

ISSN (print): 0031-8388

ISSN (electronic): 1423-0321

Publisher: De Gruyter Mouton

URL: https://doi.org/10.1515/phon-2025-0062

DOI: 10.1515/phon-2025-0062

Altmetrics

Funding

Share