[PATCH] Simulation - Game Record/Replay

[Stuart Walpole]

• Initial settings and all input is recorded.

• These same inputs can be fed back into the game at a later date so that the player can view it as a replay.

• User cannot affect what happens, but can view it (alter playback speed, look at play from any players' POV, etc).

• Needs a JS Interface. Should probably start and stop recording synchronized with CGame somehow. Needs a file format (pipe network packets to a file). Needs an entry point for replay (could do something wicked with CMessagePipe and the polling framework already in place for sockets, perhaps). http://www.wildfiregames.com/forum/index.php?showtopic=1665

• Also would need a Game Replay user interface to handle special features (selecting control of a different player, adjusting the rate of playback, rewind, etc) as well as select a replay to load (variation of Load Game screen).

---

Related tickets: #3168 Lets observer change perspective #3253 Save playernames to commands.txt #3255 Update directory structure to prevent overwrites #3258 Main menu entry to start replays #3261 Session panel (pause / fast forward / rewind) #3309 Fast forwarding replays doesn't work as expected #3387 Access summary screen from replays

[(none)]

Milestone Advanced Multiplayer Demo deleted

[historic_bruno]

Attaching a work in progress patch for visual replay that uses the existing commands.txt log files. Currently it can only be accessed from the command line:

pyrogenesis -replay-visual=\path\to\commands.txt

Current features:

• Can choose which player to view (see their LOS, resources, etc.)
• Can view selection details for any entity
• Can view production queues (research and technology) for all structures
• Can use other session features (e.g. developer overlay)
• Can not control what's going on in the game, it's only for viewing
• Notifies the user when the last replay command is reached, can optionally continue viewing

An overview of how it works:

• Similar to autostart, a new startup mode for visual replays
• CGame currently responsible for parsing commands.txt, works similarly to current non-visual replay (see Replay.cpp​)
• New turn manager, CNetReplayTurnManager stores the list of commands (as strings) and adds them after each turn
• Shares some aspects of the normal session UI but has it's own page (I thought this was better than special casing all the different logic)
• CGame detects the last command and sends a message to the replay UI

TODOs:

• Make this a main menu option
• The replay data is stored in text files which are not efficient to parse, should probably be more like saved games and include separate metadata. Compression would be nice
• No speed controls in the UI
• Doesn't really support "rewind" - rewind requires something like timewarp mode that stores the simulation state at each turn to allow going back in time, which eats lots of memory
• Make saving replay data an option?
• Use VFS to access replay file?
• Possibly factor out common parts of session and replay UI into separate files? (may also help with e.g. observer mode)

[Jonathan Waller]

Just some comments on what I think would make a good replay selection GUI. If we are auto saving every game we want to make it possible to find games easily without the user needing to enter a suitable title.

The key information is player names, date, game duration and map name. Some indication of the winner would be nice as well (maybe an icon next to their name). Giving the option to name a game manually would be good as well but I think auto naming should be primary.

[alpha123]

Update for a newer SVN version.

[alpha123]

Update for SVN r14723. May also work on A15.

[elexis]

Updated patch and removed GUI code. Replays a commands.txt file visually without GUI.

[elexis]

This patch replays a commands.txt visually with the -repay-visual=/path/to/commands.txt argument. Contrary to earlier patches, this one uses the observer mode, so that the HUD is visible and the viewer can't issue commands while replaying. Also it doesn't change the DEFAULT_TURN_LENGTH_SP to 500, but rather supports variable turn lengths by reading the turn length from the replay and updating it each turn. It seems to work fine already. However to make sure that the replay shows the actual recorded game, further testing is necessary. I thought it might be feasible to check for oos by reading the hash values from the replay file too and comparing them to the hash of the simulation state. However I get an out of sync on each turn with this version. Help welcome.

[elexis]

Thanks to Itms for committing r16531! This has helped me to find the reason why the previous patch caused an oos on every turn.

First I produced some oosdata with the -ooslog option and replayed the commands.txt after that with the -ooslog and -replay-visual option. This is the diff for the first turn:

1c1
< State hash: 5f703e921353f9018aa129164a7b4c4e
---
> State hash: 06f3356df65d0fe3e4392afbe2d3a224
28849c28849
<   "time": 500,
---
>   "time": 200,
28851c28851
<   "turnLength": 500
---
>   "turnLength": 200

Updated patch will follow soon.

[elexis]

No more out of sync. Should be ready to commit / ready for review. Other observer related features will be implemented in different tickets, like #3168.

[elexis]

JFYI: I removed the const keyword in Replay.h, because I otherwise would get the following compile error:

../../../source/ps/Game.cpp: In constructor ‘CGame::CGame(bool, bool)’:
../../../source/ps/Game.cpp:82:43: error: invalid new-expression of abstract class type ‘CDummyReplayLogger’
   m_ReplayLogger = new CDummyReplayLogger();
                                           ^
In file included from ../../../source/ps/Game.cpp:39:0:
../../../source/ps/Replay.h:55:7: note:   because the following virtual functions are pure within ‘CDummyReplayLogger’:
 class CDummyReplayLogger : public IReplayLogger
       ^
../../../source/ps/Replay.h:44:15: note: 	virtual void IReplayLogger::Turn(u32, u32, std::vector<SimulationCommand>&)
  virtual void Turn(u32 n, u32 turnLength, std::vector<SimulationCommand>& commands) = 0;
               ^
engine.make:259: recipe for target 'obj/engine_Release/Game.o' failed

On may 19th leper noted on irc why that change is needed:

(17:10:33) leper: ah, yes removing that const makes sense since the other loggers don't have it (17:11:15) leper: which means you have a pure virtual function in that the dummy logger which is what the compiler complains about (17:11:23) leper: now why are those others non-const (17:12:25) leper: could it be that one "bug" that StringifyJSON needs non-const? (17:17:07) leper: elexis: so that const change is ok (17:17:32) leper: (because JS_Stringify needs a JS::MutableHandleValue)

[Itms]

Here is a first review of the patch.

Code:

• NetTurnManager.cpp l484: remove that debug output
• l504: maybe replace that by a std::find, it's more usual
• l512: place the ENSURE here, no need to create this ok
• l525: use a reference, no need to copy that
• l526-527: remove the iterator, use for (auto& command : playerCommands) and just call command.first/second (no need to create a player variable either)

• Generally speaking, use player_id_t instead of any kind of integer for player variables

• NetTurnManager.h l25: leave that empty line between includes and class definitions
• l264: remove that space before >

• Game.cpp l148 wat?
• l150 did you figure out?
• l359 forgot the tab

• Game.h l23 leave the empty line between STL inclusions and ps inclusions

Usability: It works nicely for a dev-oriented feature, it will need a GUI, all these observer mode enhancements to be usable by standard players, of course. However, when the replay is finished, the simulation just freezes, which is not nice. The software should be closed (properly ), like for a non-visual replay.

Thanks for working on this! :)

[elexis]

Cleaner code; quits early if file doesn't exist; displays popup on oos; clean exit when finished replaying.

[elexis]

Replying to Itms:

• l504: maybe replace that by a std::find, it's more usual

std::find is not suited to test if an elements exists, since it returns the last element in that case. (You were right, that works too, fixed in the new patch)

I implemented all you asked for besides that. The code cleaning you suggested was applied to other code parts too.

New Features: -If the hash in the replay file doesnt match the simulation state hash, then the well known oos popup message will be displayed.

-If a visual or non-visual replay is started with a file that doesn't exist, then this is detected as early as possible (i.e. before starting the renderer) and the programm will be quit immediately (without crashing, as it was before using ENSURE).

-If the visual replay has finished, a global message is sent and the session.js file opens a popup box allowing to exit or continue. If you continue you will be to see the game as it was on the last turn. (Copied from one of the old patches)

It will need a GUI, all these observer mode enhancements to be usable by standard players

The other features should be implemented in other tickets, see description for links.

[elexis]

Same patch as v2, only changed 'm_ReplayHash.count(turn) == 0' to 'm_ReplayHash.find(turn) == m_ReplayHash.end()'.

[sanderd17]

The code looks good for me, but as you're changing these files, I think it would be good to take the OOS LOGERROR (NetTurnManager.cpp Line 270) out of the else. It would be handy to also get this message in the console and the logs, even if users can see it on screen.

But for me, it's ready for inclusion.

[Itms]

Hey, finally have some time to review.

It works fine here. You should fix the following:

• Change copyright dates in all the files you modified
• NetTurnManager.cpp
  • You don't need to include Player.h
  • l264 if(path)
  • new line l514
  • add brackets line 545
• Game.cpp
  • Remove that useless black magic line 57!
  • Move your two new functions elsewhere that between the ctor and the dtor, and add new lines
  • use switch in LoadReplayData
• Game.h
  • I think <map> is not needed here, if you need it in the .cpp include it there

You can also change what Sander said above.

Thanks for the patch.

[elexis]

Changes what leper suggested on june 2nd in irc and what sanderd17 and Itms commented on above. Also adds a check so that the oos-error message box will be shown once per replay at most. Switch statements with strings are not possible with c++, unless you go for ugly char array comparisons. The oos error message should be translated, but in another ticket.

[elexis]

Fixes whitespace and accidentally included premake change.

[Itms]

In 16727:

Visual replay of command files, patch by elexis.

Works with the command line argument "-replay-visual=/path/to/commands.txt". It is not integrated to the main menu GUI yet.

Refs #9.

[Itms]

Thanks for that patch. We're waiting for a cool GUI now! :D

[ben]

In 16765:

Updates readme.txt with new visual replay option, refs #9

[mimo]

In 17017:

Rename some variables to point out that they are only used in visual replay. Add script function IsVisualReplay. Refs #9 #3355, patch by elexis

[elexis]

Everybody who wants to replay games on alpha 18 can use this patch. It also contains other things like #3241. It doesn't change the simulation state of the game, so you can play in the a18 lobby as usual (tested many weeks).