[PATCH] Better font support

[sanderd17]

I don't exactly know if this should be part of the l10n branch. It isn't really dependent on it (nor the other way around), but it is somehow related, and I don't want to lose the work.

Basically, I made fontbuilder support fallback fonts. The primary and fallback fonts are mixed into one font texture which can be used by the Engine. I've experienced that, apart from packaging size, this method doesn't have many drawbacks. The textures seem to get cached on a per-glyph base. Which means that, if you don't use the huge amount of Chinese glyphs, they'll never get cached.

Only font generation time has increase a big deal, but as all Chinese glyphs have the same bounding box, the naive placement algorithm tends to use even less texture space than the smart algorithm. This is because the entire Chinese unicode block fits into a perfect grid, and takes up 90% of the total texture space. While the smart algorithm leaves this grid, and creates quite some holes.

The charset.txt, and the generator for that file have become unnecessary too. Finding out which ranges were used for which languages was too much work. Instead, all glyphs available in the font are rendered.

I also edited the GUI text renderer to mark every Chinese glyph as the end of a word. Chinese texts use no spaces, so the text was seen as one word, which didn't get divided over several lines.

As fallback fonts, I added FreeFont (​https://www.gnu.org/software/freefont/) and Hanazono Mincho (for which Wikipedia says it's available under a free license: ​http://en.wikipedia.org/wiki/Open-source_Unicode_typefaces and ​http://en.wikipedia.org/wiki/List_of_CJK_fonts )

Those fonts aren't in the patch, though they're easy to install.

There's still no support for RTL text, and it will require quite some changes in the text renderer (see ​http://www.unicode.org/reports/tr9/)

[historic_bruno]

Replying to sanderd17:

Basically, I made fontbuilder support fallback fonts. The primary and fallback fonts are mixed into one font texture which can be used by the Engine. I've experienced that, apart from packaging size, this method doesn't have many drawbacks.

What is the size difference of the new textures?

Also, you might test with Biolinum as primary font, which seemed to have the most support as a replacement for the current ugly in-game fonts: http://www.wildfiregames.com/forum/index.php?showtopic=17667 I don't know how much progress has been made on replacing the fonts.

[sanderd17]

The texture format isn't different, but adding Chinese support made them just a few magnitudes bigger. Their Unicode character range is from 0x4E00 to 0x9FFF. And those Chinese glyphs aren't small either.

So it's important to check that it doesn't cause performance issues, at least not when rendering in English.

I'll take a look at Biolinum. Btw, replacing the fonts in the GUI files isn't needed, as the GUI pages just refer to "serif", "sans", "bold" etc, and not to a specific typeface. The fontbuilder binds those typefaces to the correct names used by the GUI, so it only needs a few changes in the fontbuilder, and adding the right otf/ttf files.

This patch is mainly to see if our existing text renderer is suitable for foreign scripts, or if we need to investigate in adding a real text rendering library. As far as I've seen, the renderer seems to be quite suitable.

[sanderd17]

In 14853:

• Add support for fallback fonts in the font builder.
• Replace DejaVu with Biolinium for the existign sans-styles.
• Add additional sans-styles using Biolinum.
• Update the GUI style definitions to use the sans-styles instead of serif styles.

(Wait with updating custom defined fonts in strings, as it will cause a mess with the internationalisation that's about to be included)

• Add fonts from the quite complete FreeFont family as fallback fonts.
• Render all glyphs supported by the different fonts to support as many scripts as possible.
• Fixes #2457.

(Now hope I included all needed files)