Make XMB more efficient

[Philip Taylor]

Do various things like:

• Store strings as UTF-8
• Add GetValueAsFloat() etc to the API
• Store typed data, for more efficient int/float/etc storage. (And have something in the API to define which storage type to use for a given element/attribute.)
• Store repeated strings (e.g. entity names in maps) once in a table, instead of storing them repeatedly.

[ben]

In 15309:

Changes XMB format to store UTF-8 strings and removes pointless UTF-16 conversions, refs #204, #244.
Bumps XMB version to 2.

[TheCake91]

While trying to work on this ticket I started studying the associated code (XeroXMB.cpp). I quickly started wondering why int is used instead of a fixed width type like u32 for meta values (e.g. ElementNameCount) in the binary file? Wouldn't this cause some "theoretical" inconsistency across different architectures? Although I do not expect that a user actually will exchange these files it might be nice to have a file format that is consistent across different architectures.

[Stan]

As the name suggest u32 is an uint32_t aka an unsigned int. uint32_t is not a standard. It's usually defined but we can't assume it is. So that's why we use it.

As for unsigned vs signed I guess elementnamecount should always be positive. The issue is the following the game is packaged on a 64bit Debian machine. Which means XMB have the properties of that machine if another machine interpret stuff differently it will break.

[TheCake91]

Replying to stanislas69:

t's usually defined but we can't assume it is. So that's why we use it.

There is also the possibility using a part of the boost library which defines these fixed width types based on various conditions ​https://www.boost.org/doc/libs/1_69_0/boost/cstdint.hpp. Although as long as we do not intend compiling the code for exotic embedded system or special purpose microprocessors it seems unlikely that the compiler won't support these optional types.

But I guess it is not that important either and trying to gauarantee binary compatibility across various architectures would probably cause even more difficulties. Especially when returning those values as int after deserialization.

[wraitii]

r25375 made the format more efficient by storing names as offsets, which made using name-as-string more efficient.

The rest is entirely unchanged at the moment.

See also #244