Except many many experts have said this is not why it happens. It cannot count letters in the incoming words. It doesn’t even know what “words” are. It has abstracted tokens by the time it’s being run through the model.

It’s more like you don’t know the word strawberry, and instead you see: How many 'r’s in 🍓?

And you respond with nonsense, because the relation between ‘r’ and 🍓 is nonsensical.