Automatic checksumming and download of needed files

[Simon Brenner]

Brought up in the Development Notepad

From the (old) TDD: (http://www.wildfiregames.com/forum/index.php?showtopic=597)

Custom maps, and all resources they may require, should be synchronized to all
clients before entering the in-game state. Each client receives a list of files
that are required by the map, with checksums and sizes – it is then up to the
client to tell the server which files require downloading.

The Pre-Game state does not end until all clients have downloaded all required
files from the server, and the server then sends a Start Game message. Each
client sends a list of files to the server, the server then begins to send all
files to all clients requiring downloads in chunks. To get a synchronous
behavior, clients acknowledge the receipt of each chunk (despite the fact that
we’re using reliable, ordered communication). At a fixed, quite large (at least
one second), interval, the server sends File Transfer Progress messages to all
clients, so that they may display the progress to the user.

When all files have been transferred, the server sends the Start Game message
to proceed into the in-game state and start the actual game.

What we'd need to do, which we don't have code for yet, is:

• Enumerate all files required by the map. This might well be the hardest part - many of our structures are AFAIK built from *all* files, regardless of whether they're actually referenced by the map - but it would also be useful/required for automatically creating self-contained custom maps.
• Produce checksums for all files
• Compare checksums to the list given by the server
• Decide on a place to download it all, and create a folder for it
• Download stuff. Networking would require a few extensions to the socket classes to support file transfers well (Packets are only added to the send queue once per frame, so either file transfers are limited by frame rate, or the other packets would be blocked by all the file transfer packets)
• Recheck the checksums. If nothing else, to check the network code for bugs :P