Split the POT file of the public mod in more manageable POTs

[Adrián Chaves]

Reasons

We should do this because it makes the life of translators easier:

• It gives them an implicit context, as each POT will only have strings pertaining to certain areas of the game.
• It helps them focus on the strings that matter the most. For example, we could have a POT file for GUI strings, and translators could try to keep that at 100% first, before they start working on other translations.

Deciding How to Split

Criteria for the split:

• We should aim for POT files with less than 1000 strings each.
• We should try to keep POT files specific to a topic. For example, a POT for the GUI, a POT for map descriptions, a POT for help (e.g. manual), etc.

We should also decide on the barrier for languages to enter the game based on the splitted POT files. Do we want all files to be at 90%? Do we want some at 100%? Do we care if some of them are at 0%, 25%, 50%, or 75%?

Changes to the l10n Folder Structure

In order to avoid having hundreds of files around in the l10n folder, once we split the POT files we should have a subfolder of l10n for each POT file:

• l10n
  • messages.json
  • public.gui
    • template.pot
    • de.po
    • …
  • public.maps
    • template.pot
    • gl.po
    • …

As you see, having different folders allows us to use simpler names for the actual POT and PO files, as the subfolders should be enough to ensure that there is no conflict with other mods.

Implementation

Before the Split

1. Decide which POT files we will have and what content will each of them contain.

2. Ensure that both the engine (C++) internationalization and our internationalization tools (e.g. message extraction) work OK with subfolders in l10n and the proposed file names.

3. Create the scripts required for the change:

• A script to generate the splitted PO files from the existing PO files once the splitted POT files are ready. (See the expected logic below)
• A script to upload the splitted translations to Transifex.

4. Announce in Transifex which day this will happen (it can be announced a couple of days before). During the split, we will announce when the process starts (translators should not work on translations from then on) and then announce it once it is finished and translators can start working on the new resources.

Script to Generate Splitted PO Files

Logic in pseudo-code:

for each original PO file:
    for each POT file:
        Copy the PO file into the folder where the POT file is, with the new PO name (e.g. de.po).
        Update the PO file using the POT file. `msgmerge -U de.po template.pot` leaves old translations behind, which we do not want in a split. In https://drupal.org/node/11311 they have a suggestion (--compendium) that might be what we need.

Steps for the Split

1. Generate the new POT files.
2. Announce the start of the task in Transifex.
3. Pull the latest translations from Transifex.
4. Create the new resources in Transifex.
5. Run the script to generate the splitted PO files.
6. Delete the old PO and POT files (locally only).
7. Add config files (.tx/config) for the Transifex client to the new l10n subfolders that match them to a resource in Transifex.
8. Run the script that uploads the new PO files to Transifex.
9. Announce the end of the task in Transifex, as well as the new requirements for translations to be distributed with the game (if there have been changes in this regard).

[sanderd17]

IMO, the name of the mod (public) should stay part of the po file name. We can't allow the public mod to split the po files per context, while enforcing other mods to cram everything in one file, as eventually, the topics of different mods will collide.

So there are a quite some possible naming conventions left:

lg.modname.topic.po
modname/lg.topic.po
topic/lg.modname.po
topic/modname/lg.po
...

I also don't know what the problem is with many files in one directory. It's not like our current directories are that small (like ​http://trac.wildfiregames.com/browser/ps/trunk/binaries/data/mods/public/simulation/templates/units). I think it's a bit better to have it all in one directory, as it allows for easier bash operations, and it needs almost no adaptation of our tools.

I do agree with splitting the files for better quality control (the quotes don't have to be translated, but the GUI does).

[Adrián Chaves]

If having many files in the same folder is not an issue, I’m OK with “lg.modname.topic.po”.

[Adrián Chaves]

We’ve discussed in the forums how to split the translation templates: http://www.wildfiregames.com/forum/index.php?showtopic=18789

We’ve agreed on Sander’s approach.

[gallaecio]

In 15397:

Splitted the PO and POT files

Fixes #2494

[ben]

In 15800:

Fixes regex for excluding translations from release packages, refs #2494