The Case of the Trailing Қ
I’m building a local handwriting recognition tool for scanned notes in English and Tajik. The workflow is straightforward: crop a line, straighten it, transcribe it, and use those corrections to fine-tune a model.
Then my model developed a peculiar habit.
It started giving English sentences a Tajik farewell:
to the progress of our knowledge theқ
That final қ is a perfectly legitimate Tajik Cyrillic letter. It just has no business standing at the end of that English sentence.
Was it the UI?
My first question was whether the UI was accidentally appending it. A string concatenation bug?
Then I started looking directly with the fine-tuned model. The saved prediction files answered that: the character appeared before predictions were integrated into the UI.
On our 29 English test lines:
| Model and input | Lines ending in қ |
|---|---|
| Pretrained model, original scans | 0 |
| Pretrained model, isolated ink | 0 |
| Fine-tuned model, original scans | 15 |
| Fine-tuned model, isolated ink | 0 |
The annotation data wasn’t responsible either. No English training transcript contained қ, and no training transcript ended with it.
The model had learned this little signature all by itself.
A bilingual model with an uneven education
I had fine-tuned a multilingual handwriting recognizer on a small dataset:
- 129 Tajik training lines
- 24 English training lines
The pretrained model lacked some Tajik-specific characters I needed, so I expanded its output vocabulary before fine-tuning.
That worked: the model began producing Tajik characters. Tajik character error rate on the original test crops dropped from 23.5% to 17.5%.
But the English output acquired an unexpected behaviour.
The imbalance is a plausible contributor, though it doesn’t fully explain why қ appears specifically at line endings—and only with original scans in this test. That pattern suggests the paper background or crop-edge appearance may also be involved. I haven’t confirmed the exact trigger yet.
Better overall doesn’t mean better everywhere
Overall character error rate improved from 19.2% to 16.6%. A single summary score would make the experiment look like a straightforward success.
Looking at the languages separately told a more interesting story. Tajik improved substantially, while English word error rate worsened.
A model can improve on average and still develop a very visible bad habit.
What comes next?
There’s more to investigate. My next step is to add more English pages and reduce the imbalance in the fine-tuning data. Since I already label each line by language, I also want to experiment with language-aware decoding.
Simply removing every trailing қ would hide the symptom. I’d rather understand why it appears—and make sure fixing it doesn’t just replace it with a different wrong character.
For now, it’s a useful reminder that fine-tuning isn’t a magic “make it better” approach.
Sometimes it teaches your model another language.
Sometimes it teaches it to end English sentences with қ.