Pronunciada

Open source · Apache-2.0

On-device pronunciation assessment with cloud-level accuracy.

Pronunciada is an open-source Python library that grades a language learner's speech sound by sound, in ten languages.

Source on GitHub pip install pronunciada
77 of 84comparisons level with or ahead of Azure
50 msper sentence on a laptop CPU
10languages

Tested against Azure in ten languages

Most language-learning apps get their pronunciation scores from a cloud service, and the most widely used one is Microsoft Azure Pronunciation Assessment. Pronunciada was compared with it in every language it supports: English, Mandarin, Japanese, Spanish, French, German, Italian, Portuguese, Russian and Korean. There are 84 comparisons. Each is one measure on one public test set, with the same recordings and text given to both systems.

Ten languages on one shared speech model, with a small adapter for each.

Comparisons with Azure, by language
Pronunciada significantly betterNo significant differenceAzure significantly better
English1153of 19
Mandarin511of 16
Japanese312of 6
Spanish321of 6
French51of 6
German34of 7
Italian33of 6
Portuguese321of 6
Russian33of 6
Korean15of 6

84 comparisons on held-out public test sets. Significance is from paired tests at p < 0.05.

Pronunciada is significantly better in 40 comparisons and Azure in 7. The other 37 show no significant difference. Azure's seven are English intonation, how often a native speaker's words are marked down in English, Japanese, Spanish and Portuguese, and two tests on Common Voice recordings. Every comparison is listed in the scoreboard.

Closer to expert raters on learner speech

The most direct test uses recordings of learners that experts have scored by hand. Such recordings are public for English and Mandarin. On them, Pronunciada agrees with the experts more closely than Azure on individual sounds, on whole sentences and on Mandarin tones. Azure is closer on English intonation.

Learner speech scored by experts
PronunciadaAzure
English soundsagreement with expert scores
0.69
0.54
English sentencesagreement with expert scores
0.75
0.67
English mispronunciationsfinding the sounds experts marked
0.71
0.63
Mandarin tonesfinding the tones teachers marked
0.78
0.68
English intonationagreement with expert scores
0.77
0.83

Higher is better. Agreement is the correlation with expert scores, and finding errors is the area under the ROC curve. Test sets: speechocean762, L2-ARCTIC and OMPAL. All five differences are statistically significant.

All figures come from held-out test recordings. Azure was used as provided, through its REST API in November 2026.

Runs on the device in about 50 ms

Scoring needs no network connection and no GPU, and the audio does not leave the machine. A sentence of 3.5 seconds is scored in about 50 ms on an Apple M3 Pro laptop.

The system was designed with phones in mind. The quantized model is a 67 MB file, runs on a single CPU thread and uses about 0.4 GB of memory. The full model is 248 MB and uses about 0.6 GB. Packaged iOS and Android builds are not available yet.

Time to score a 9-second recording
Pronunciadaon the laptop, one CPU thread0.30 s
Azurerequest to the cloud and back2.34 s

Lower is better. Median over English recordings. This is not a controlled comparison: the two sets of recordings differ, and the Azure time is the whole request from the same laptop, upload included.

Built on ZIPA, and far more accurate than ZIPA alone

No large model was trained for this project. The sounds in a recording are recognized by ZIPA (Zhu et al., ACL 2025), an open speech model trained on many languages, and Pronunciada adds the scoring around it. To measure what that adds, the same tests were run on ZIPA alone with the standard scoring method.

The same tests, against ZIPA alone
PronunciadaZIPA alone
English soundsagreement with expert scores
0.69
0.43
English sentencesagreement with expert scores
0.75
0.58
English mispronunciationsfinding the sounds experts marked
0.71
0.66
Mandarin tonesfinding the tones teachers marked
0.78
0.55

Higher is better. ZIPA alone is the unmodified model with the standard goodness-of-pronunciation score. All four differences are statistically significant.

Across 60 tests, Pronunciada is significantly better in 45 and significantly worse in none. The gap is widest for Mandarin tones, which ZIPA does not output. The full comparison is in the baseline write-up.

How it works

Pronunciada compares the sounds a sentence should contain with the sounds a speech model hears in the recording. The figure follows one word from the learner recording in the film above.

How one word is scored
“float”, said without its “l”
Expected soundfloʊtRecognized instead–nothingɔ–Probability a rater accepts it0.830.270.350.56

The speech model heard no “l” and a different vowel, so those two sounds score low and the word as a whole gets 0.10.

  1. The text is converted to the sequence of sounds a speaker should produce, using a pronunciation dictionary and rules for each language.
  2. ZIPA listens to the recording and reports which sounds it hears, moment by moment, without seeing the text. A small adapter per language corrects its systematic confusions.
  3. For every expected sound, the scorer weighs three explanations: it was said, it was replaced by another sound, or it was dropped. It sums over every way of lining the sounds up with the audio, so no single alignment decides the result.
  4. Calibration models fitted on expert-scored speech turn that evidence into the probability that a rater would accept each sound, each word and the sentence. One strictness setting moves all of them together.
  5. Separate small models handle what the sequence of sounds cannot show: tone, stress, length and a few consonant contrasts.

The models trained for this project add about 10 MB to the package. The design document has the details.

Limitations

The README has the full list.

Get started

pip install pronunciada
import pronunciada

result = pronunciada.score(
    "clip.wav", "It was a beautiful day for a float", lang="en"
)

result.prob                # 0.75, the sentence
result.words[7].prob       # 0.10, the word "float"
result.words[7].phones     # f 0.83, l 0.27, oʊ 0.35, t 0.56
result.coaching()          # detected errors, most severe first

The values are for the learner recording in the film above, in which the “l” of “float” is dropped. The model is downloaded on first use. A command-line tool, a local HTTP server, output in Azure's JSON layout and an MCP server are documented in the README.

Project

Maintainer
Bill Sun
License
Apache-2.0 for the code. ZIPA and the sample recordings are licensed under CC BY 4.0.
Built on
ZIPA (Zhu et al., ACL 2025) and ONNX Runtime
Test data
speechocean762, L2-ARCTIC, EpaDB, OMPAL, FLEURS and Common Voice
Citing
CITATION.cff in the repository
Contributing
Issues and pull requests are welcome on GitHub. See CONTRIBUTING.md.

Apache-2.0, on GitHub

The code, every result table and the scripts that produce them are in the repository.