John McWhorter:

In Latin, H was “ha,” and K as “ka” came after it. People reciting the alphabet would say “ha, ka, el ….” But Latin didn’t really need the K, because C did its job. (Caesar was pronounced “KIGH-sar.”) So people started thinking of “ha, ka” as referring just to H and saying “haka.” Heavy usage transforms pronunciation. “Haka” dropped the “H” at the start like Eliza Doolittle was taught out of in her “Hertford, Hereford and Hampshire” exercise — “aka.” Then the K changed to a “ch” — “ahcha” — and the A at the end dropped off — “ahch.” Then the “ah” changed to “ay” and hence “aitch”! We should really be calling H something like “hay.” But hey, we don’t — and the world keeps spinning.