mojibake on pages in zh-HK locale with special symbols
When I customise fonts using Firefox settings (for zh-CN, zh-TW, zh-HK and jp locales) and choose to override fonts specified on a page, some special symbols on pages in the zh-HK locale are not rendered correctly.
https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-26-04-51-37-c4ca42.png
this happens specifically for the zh-HK locale. If I switch to zh-TW, they are rendered correctly: https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-26-04-52-48-c81e72.png
I looked into the problem using "inspect element" and checked the "fonts" tab. It seems special fonts are used in the page. In a page in the zh-TW locale, the fonts are correctly loaded as: https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-26-04-56-14-eccbc8.png
with zh-HK, some font on my system (MingLiu_HKSCS) is used as fallback: https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-26-04-59-34-a87ff6.png
There are pages that are only available in zh-HK. I would like to use my preferred fonts (as in other locales such as zh-TW) but this creates mojibake on the page.
How could I solve this problem?
All Replies (12)
Hi Glen, I don't fully understand how fonts work, but I wanted to check whether you have set your preferred font for both the HK and TW character sets in the Fonts dialog, because Firefox seems to have separate font lists for each one.
You can check that here:
- Windows: "3-bar" menu button (or Tools menu) > Options
- Mac: "3-bar" menu button (or Firefox menu) > Preferences
- Linux: "3-bar" menu button (or Edit menu) > Preferences
- Any system: type or paste about:preferences into the address bar and press Enter/Return to load it
Scroll down to the Language and Appearance, Fonts and Colors section, and click the Advanced button. Use the selector at the top to change between all the relevant character sets and verify that they list the fonts you want:
Chinese characters include a lot of Unicode code points. If you disable website fonts then you won't see special icon fonts used by the website, but you may see a Chinese character instead if you have a font that covers this glyph.
It is always best to allow website to use their own fonts to avoid such issues.
i have set my preferred fonts for simplified chinese, traditional chinese (hong kong), traditional chinese (taiwan) and japanese
Google offers regional variants of Noto Sans for all East Asian writing systems, so I use the specific variant for the writing system that it is designed for
so the interesting thing is that my setting only fails for zh-HK
the same custom fonts are used in this website for all jscher2000 said
Hi Glen, I don't fully understand how fonts work, but I wanted to check whether you have set your preferred font for both the HK and TW character sets in the Fonts dialog, because Firefox seems to have separate font lists for each one. You can check that here:Scroll down to the Language and Appearance, Fonts and Colors section, and click the Advanced button. Use the selector at the top to change between all the relevant character sets and verify that they list the fonts you want:
- Windows: "3-bar" menu button (or Tools menu) > Options
- Mac: "3-bar" menu button (or Firefox menu) > Preferences
- Linux: "3-bar" menu button (or Edit menu) > Preferences
- Any system: type or paste about:preferences into the address bar and press Enter/Return to load it
(sorry i don't know how to embed images in my post)
this is my setting for the three chinese variants
https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-27-03-09-06-545e30.png
https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-27-03-09-10-190157.png
https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-27-03-09-14-e1ca34.png
all use Source Han Sans (just another name for Noto Sans) custom icon fonts are used in 3 language variants of the same webpage
yet only zh-HK fails this feels weird
cor-el said
Chinese characters include a lot of Unicode code points. If you disable website fonts then you won't see special icon fonts used by the website, but you may see a Chinese character instead if you have a font that covers this glyph. It is always best to allow website to use their own fonts to avoid such issues.
font rendering in Windows 10 with the default Chinese fonts is a pain for my eyes If possible I'd prefer using custom fonts
in addition, custom fonts work well with the other two Chinese variants (zh-TW and zh-CN) so I really hope to have a solution for zh-HK
I'm puzzled that webfonts are used with two of the three character sets and not the third. Could you share an example URL where you see this problem?
jscher2000 said
I'm puzzled that webfonts are used with two of the three character sets and not the third. Could you share an example URL where you see this problem?
May I ask what webfonts are you referring to?
I was only showing you two language variants because I thought that would suffice. rendering works properly for zh-TW and zh-CN and I picked zh-TW randomly.
the screenshots are from this website: https://www.office.com/?omkt=zh-HK&auth=1 https://www.office.com/?omkt=zh-TW&auth=1
(and https://www.office.com/?omkt=zh-CN&auth=1 if you are interested)
An gyara
I realised that the office.com page requires an active office subscription
here is another page: https://www.microsoft.com/zh-hk (the hong kong chinese version of microsoft.com)
the upper right corner is wrongly rendered as: https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-27-09-20-04-c498c3.png
If I visit the zh-CN version or zh-TW version, the symbols are correctly displayed: https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-27-09-20-54-5db8c2.png
An gyara
I logged in on the omkt=zh-HK URL using my @outlook.com address and turned off page fonts. Using only the fonts on my system, I got the expected icons. (Attached)
I notice I'm being served Unicode (UTF-8). On the View menu, Text Encoding is grayed, so I can't switch it to zh-HK, if that is something necessary to replicate the problem.
I didn't install the Adobe fonts you're using. That might be needed to encounter the issue.
jscher2000 said
I logged in on the omkt=zh-HK URL using my @outlook.com address and turned off page fonts. Using only the fonts on my system, I got the expected icons. (Attached) I notice I'm being served Unicode (UTF-8). On the View menu, Text Encoding is grayed, so I can't switch it to zh-HK, if that is something necessary to replicate the problem. I didn't install the Adobe fonts you're using. That might be needed to encounter the issue.
Thank you for your help, jscher2000
The pages are also served to me in UTF-8 for all 3 language variants
I tried copying your font settings (https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-27-09-36-31-52d2a7.png) and the problem still persists for me :/
https://user-media-prod-cdn.itsre-sumo.mozilla.net/uploads/images/2019-10-27-09-38-55-a052ae.png
An gyara
Thanks for the store link. With default fonts, the icon is not an available character, and after initially displaying the "missing character" box the icon loads:
<center></center>Perhaps the problem is that the Adobe font is not missing that character, so Firefox doesn't continue on to use the icon font??
An gyara
jscher2000 said
Thanks for the store link. With default fonts, the icon is not an available character, and after initially displaying the "missing character" box the icon loads: <center></center>Perhaps the problem is that the Adobe font is not missing that character, so Firefox doesn't continue on to use the icon font??
Thanks again
Mmmm I am getting some inspiration from your answer
I can see a 0xE7BF character in your screenshot
firstly, I found that 0xE7BF is a private use Unicode character
I entered 0xE7BF in Word and found that if I use the serif font MingLiu_HKSCS (but not Source Han Sans), voilà, I have exactly the same mojibake that I have on my Firefox
So it seems that my Firefox thinks MingLiu_HKSCS has the character that it needs so it ignores the custom font provided in the page
while your Firefox correctly ignores MingLiu_HKSCS (do you have that?) and goes on to load the custom font (even though we have the same font settings now)
(but why MingLiu_HKSCS?)
An gyara