Why Two Pronunciation Scores Differ: Recording Conditions, Speech Samples, and Model Variability

Before you interpret the score, ask: what changed?

If several boxes changed, you do not have a clean pronunciation comparison yet. You have two different experiments.

1. Recording conditions can move the measurement before your pronunciation changes

This is the least glamorous cause and one of the easiest to test. A microphone farther away, a noisier room, a clipped start, another person talking nearby, or a different device can change the audio that reaches the system.

Microsoft's current Pronunciation Assessment limitations explicitly say the system works better with higher-quality audio, recommends input at 16 kHz or higher, advises keeping the speaker close to the microphone, supports one speaker per assessment, and performs better with little background noise. It also recommends normal speaking speed and volume.

That documentation describes Microsoft’s system, not every pronunciation app. But it proves the broader point without any model gossip: real automated pronunciation systems can be sensitive to recording conditions.

So if attempt one was recorded on your phone in a quiet bedroom and attempt two on a laptop beside a fan, do not immediately write a tragedy about your vowels. First make the audio comparable.

Fast recording check

• Play both recordings back.
• Listen for clipped first or last words.
• Check whether one take is much quieter or farther from the mic.
• Notice background voices, echo, keyboard clicks, traffic, or fan noise.
• Retest with the same device and mic position.

2. A different speech sample is not the same test with new words

“My first sentence scored 88 and my second scored 74” sounds like a comparison. Often it is not.

The second sentence may contain sounds you find harder, denser consonant clusters, different stress patterns, more reductions, or simply more opportunities to hesitate. A five-word phrase and a 40-second answer also give a scoring system different amounts and kinds of evidence.

For a clean retest, use the same text. If you want to test transfer later, deliberately introduce a new sentence—but call it a transfer test, not a repeated measurement.

3. Reading and spontaneous speaking are different tasks

Reading gives you the words. Spontaneous speaking makes you choose the words while also planning grammar, meaning, timing, and pronunciation. Comparing those scores as if only pronunciation changed is shaky.

Microsoft's current Pronunciation Assessment tool documentation separates a Reading mode with reference text from an unscripted Speaking mode based on a topic. It also recommends a minimum amount of speech for some unscripted/content assessment outputs. That is a useful reminder: task design and sample amount are part of the measurement context.

If your read-aloud score is higher than your spontaneous score, that does not prove you “lost” pronunciation ability in conversation. It tells you the tasks placed different demands on you—and possibly asked the scoring system to evaluate different evidence.

4. You are variable too—and that is not a diagnosis

Your second take can genuinely sound different from your first. You may slow down, exaggerate a corrected sound, rush after seeing a low score, imitate the model more closely, or simply become more familiar with the sentence.

That means repeated attempts can mix learning with test practice. A fifth take after four immediate replays is not the same performance situation as a cold first take the next morning.

For troubleshooting, standardize the human side too:

• use the same short warm-up;
• keep the same speaking instruction—natural pace, not “score-maximizing voice”;
• avoid comparing a cold attempt with a heavily rehearsed one;
• if timing matters, test at roughly the same point in your practice session.

You are not trying to become a laboratory robot. You are trying to stop changing five variables and blaming one.

5. Reliability and validity are not the same question

These two words sound academic. The distinction is actually practical.

Reliability asks: if conditions stay similar, does the measurement behave consistently enough to be useful?

Validity asks: does the score meaningfully represent the thing you care about?

A score can be reasonably repeatable and still represent only a bounded slice of pronunciation. A ruler can be very reliable at measuring height; that does not make it a valid measure of athletic ability.

That is why the sentence “the app gave me 91, therefore my pronunciation is 91% correct” usually says more than the evidence supports unless the product clearly defines what 91 represents.

Peer-reviewed work makes this boundary concrete. Cai and colleagues (2025) built and validated a specific automatic pronunciation scorer around explicitly chosen pronunciation and intelligibility-related constructs and examined bias. The point is not that every app works like that scorer. The point is the opposite: what a score means depends on what the scoring system was designed and validated to measure.

6. Automated systems can disagree without one revealing the secret truth

Do not try to explain every score swing by inventing a hidden formula: “Maybe this app weights /r/ at 17%, then subtracts fluency if I pause after articles.” Unless the vendor documents that, it is fiction with a spreadsheet costume.

A 2025 study by Yongkook Won compared human comprehensibility and accentedness ratings with outputs from six ASR/pronunciation systems on 190 read-aloud recordings by Korean elementary learners. The relationships differed by system and metric; some systems with very low word error rates still showed weak relationships with human pronunciation judgments in that bounded dataset.

Do not turn that study into a ranking of today's apps. Its learners, task, systems, and versions were specific. The useful lesson is narrower: different automated measurements can behave differently, and transcription success is not automatically human intelligibility.

The three-take controlled retest

If you actually want to know whether a score change is meaningful enough to investigate, run this before changing your pronunciation plan.

1. Record take one.
2. Wait briefly. Do not stare at one flagged phoneme and remodel your entire speech.
3. Record take two under the same conditions.
4. Record take three.
5. Write down all three scores—not only the best one.
6. Play the recordings back and note any obvious audio or performance difference.

There is no defensible universal rule that says “a difference of X points is normal” across all pronunciation products. The scale, construct, task, and score aggregation can differ. Look for a pattern inside your controlled setup instead.

The Score-Debugging Lab

Choose what you would control first, then open the answer.

When to stop chasing the score and check human intelligibility

The score is a tool. Communication is the job.

Escalate beyond the number when:

• controlled scores stay unstable enough that they are not helping you choose practice targets;
• two systems repeatedly disagree;
• your score is stable but real listeners often ask you to repeat the same word or phrase;
• your score drops but listeners understand you easily and your recordings sound clear in context.

A useful human check is simple: ask a listener who has not seen the script to write what they heard, or ask whether one target sentence was easy to understand without extra effort. That does not create a perfect human “truth score” either. It answers a different, very important question: did the message get through?

Practise the behavior, not the number

Once your controlled retest reveals a repeatable behavior—perhaps a word disappears at normal speed, a contrast collapses in a sentence, or your rhythm becomes choppy—practise that behavior directly.

FunFluen can support that shift from score-chasing to contextual speaking practice. Choose a speaking-practice path and practise the behavior, not the score. The exact issue is not preloaded, and FunFluen is not a universal pronunciation authority or ground-truth scorer.

If you want to work with authentic language in context, the broader media-based language learning hub is the better next step.

The rule worth keeping

A pronunciation score is a photograph, not a passport. If you want to compare two photographs, use the same lighting.

Hold INPUT, SAMPLE, TASK, and test setup steady. Record several takes. Look for a pattern. Then ask whether the score's construct matches the communication skill you actually care about.

Track a series, not a selfie—and never let a precise-looking number bully you into an explanation the evidence cannot support.

Sources

• Microsoft: Characteristics and limitations of Pronunciation Assessment
• Microsoft: How to use pronunciation assessment in the Foundry portal
• Won (2025): Assessing the efficacy of word error rate as a proxy for pronunciation quality
• Cai et al. (2025): Developing an Automatic Pronunciation Scorer