#3258 closed enhancement (fixed)
[PATCH] Visual Replay Menu
Reported by: | elexis | Owned by: | Itms |
---|---|---|---|
Priority: | Should Have | Milestone: | Alpha 19 |
Component: | UI & Simulation | Keywords: | patch |
Cc: | Patch: |
Description (last modified by )
0ad will have the functionality to replay a game visually (see #9). To make this feature accessible to all users, a menu should be added that allows the user to select and start replays.
Features:
A new main menu item called "Replay" should be placed below the Singleplayer and Multiplayer buttons.
If clicked, it should show a list with selectable items, one for each replay file (commands.txt) found, similar to the Singleplayer->LoadGame
dialog.
If an item is selected, it should display useful information of that match, including date, time, playernames, map thumbnail, victory condition and population count.
The dialog should have buttons "Cancel", "Delete" and "Play".
Also another text input line would be great, where you can enter some arbitrary strings, like playernames, mapnames, victory condition in order to filter the items displayed and find particular games quickly.
Technical Requirements:
The commands.txt must contain the engine version so that it can be checked explicitly for compability before replaying the file. It should also contain a timestamp, so that the date value won't be lost if players share their files over the internet.
The engine version is currently saved in XmppClient.cpp
and should be moved to another file. Refs #2745.
m_client->disco()->setVersion("Pyrogenesis", "0.0.19");
Attachments (22)
Change History (43)
comment:1 by , 9 years ago
Description: | modified (diff) |
---|
by , 9 years ago
Attachment: | replay_menu_wip_v1.patch added |
---|
comment:2 by , 9 years ago
Keywords: | patch added |
---|---|
Priority: | Nice to Have → Should Have |
Summary: | Visual Replay Menu → [PATCH] Visual Replay Menu |
by , 9 years ago
Attachment: | screenshot_v1.png added |
---|
by , 9 years ago
Attachment: | replay_menu_wip_v2.patch added |
---|
Works for r16777. Implements a nice list, filter capabilities and preview data. Remaining Todo: requires a check so that replays of old versions will be greyed out and not replayed. Some more data like players that resigned will be displayed if a spoiler button is pressed.
comment:3 by , 9 years ago
screenshot_v2.png: https://i.imgur.com/B3XIvFR.jpg
As you can see, visual replay really needs #3253.
by , 9 years ago
Attachment: | replay_menu_wip_v3.patch added |
---|
Now lists are sortable. Thanks to vlad and sander for making #2405 happen. Added error message for corrupted files. Found out that 0ad crashes (because the the ifstream opening the commands.txt files is not good) if you open the menu the second time and I don't know why yet... Screenshot: https://i.imgur.com/4ImuMrQ.jpg
by , 9 years ago
Attachment: | replay_menu_wip_v4_16795.patch added |
---|
Fixed the crash (std::ifstream wasn't closed). After a replay, the summary screen now returns back to the replay selection screen.
by , 9 years ago
Attachment: | replay_menu_wip_v5_r16812.patch added |
---|
comment:4 by , 9 years ago
This version has no known bugs and should be commitable. Unless you stop me though, I will do another version that makes the summary screen available from the menu directly by saving the summary screen stats at the end of the game.
General Enhancements in version 5:
-moved engine version string to ps/EngineVersion.h
-refactored VisualReplay::LoadReplayData
-parses json in c++ instead of js
-saves timestamp to replay (uses file modification date as fallback)
-saves engine version to savegames and replays to add compatibility check
Tanslation Fixes:
-getReplayDateTime
now uses Engine.FormatMillisecondsIntoDateString
-getReplayDuration
now uses `timeToString
-initDurationFilter
now uses sprintf
and translateWithContext
Whitespace Fixes: -changed some instances of "for (" to "for("
GUI: -adds compability filter and greys out incompatible replays. -two popup boxes informing the player if the replay is of an outdated version of 0ad or if the user only has to enable some mods -shows more replay info: victory condition, teams, civs, player colors, resigned players (with spoiler alert checkbox), AI details -displays filename in delete dialog (where else could the filename be displayed without having to click on delete?)
Other:
-continues to use FileExists
instead of VfsFileExists
, because filenames are given as OsPath
, i.e. an absolute pathname.
Please update http://trac.wildfiregames.com/wiki/ReleaseProcessDraft and replace XmppClient.cpp
with EngineVersion.h
after committing.
by , 9 years ago
Attachment: | replay_menu_wip_v6_r16812.patch added |
---|
Saves the data that is passed to the summary screen to the commands.txt
directory. Adds a link to the visual replay menu that allows browsing the summary screen (without replaying the whole game), so that you can see who won and how. Translates some more strings TODO: the game should remember all defeated players, not only those that resign
by , 9 years ago
Attachment: | replay_menu_v7_r16841.patch added |
---|
Now displays defeated players if spoiler checkbox is checked. Some refactoring (most notably moved ai difficulty names to a separate file). Some performance optimization. Renamed replay.js to replaymenu.js (and other related files).
comment:5 by , 9 years ago
Keywords: | review added |
---|---|
Milestone: | Backlog → Alpha 19 |
Notice some new files:
svn add binaries/data/mods/public/gui/page_replaymenu.xml svn add binaries/data/mods/public/gui/replaymenu/ svn add binaries/data/mods/public/gui/aiconfig/difficulties.json svn add source/ps/VisualReplay.cpp svn add source/ps/VisualReplay.h svn add source/ps/EngineVersion.h
Don't forget to remove them after reverting the patch, otherwise you won't be able to compile and after applying the patch again the files will be messed up.
by , 9 years ago
Attachment: | replay_menu_v8_16864.patch added |
---|
comment:6 by , 9 years ago
New in version 8:
Performance Enhancements:
- reduced filter initialization complexity, now O(1) instead of O(n)
- huge memory improvement after removing array copies in
loadReplays()
Refactoring:
- moved constants from
gamesetup.js
,aiconfig.js
andlobby.js
to translatable json files - moved shared translation functions to
functions_utility.js
- uses
const
andlet
whereever possible
Visual Replay Enhancements:
- doesn't create empty commands.txt files anymore when returning from the gamesetup (by moving the file creation from
CReplayLogger
ctor toCReplayLogger::StartGame
)
- fixes a bug where the
ReplayFinished
message was not being sent (when doing too fast forwarding on a laggy game caused by #3309) by moving the sending of that message toCNetReplayTurnManager::NotifyFinishedUpdate
Miscellaneous:
- many translation fixes
- correct handling of null in
PlayerData
array - removed trailing whitespace & some useless comments
Remember to delete new files after reverting the patch:
rm binaries/data/mods/public/gui/page_replaymenu.xml rm binaries/data/mods/public/gui/replaymenu/* rm binaries/data/mods/public/simulation/ai/difficulties.json rm binaries/data/mods/public/simulation/data/map_types.json rm binaries/data/mods/public/simulation/data/population_capacity.json rm source/ps/VisualReplay.cpp rm source/ps/VisualReplay.h rm source/ps/EngineVersion.h
by , 9 years ago
Attachment: | replay_menu_v8.2_16877.patch added |
---|
Pausing replays works again, refs r16871.
comment:7 by , 9 years ago
Now supports all resolutions (1024x768 and greater). Adds clipping to COList.cpp
to prevent overwriting neighboring text.
Example 1024 x 768: http://i.imgur.com/xuFwSQ4.png Example 1600 x 1200: http://i.imgur.com/wdvWmDh.jpg
Use this command to change the size of your window. If you chose a size bigger than your current resolution, you have to move the window or make a screenshot to see all areas.
wmctrl -r "0 A.D." -e 0,0,0,1600,1200
comment:8 by , 9 years ago
To reduce the size of the patch from 140kb to 60kb, it was split into several parts:
r16895: Add clipping to OList cells (#3353)
r16942, r16975, r16979, r17006, r17007 r17012, r17017, r17019, r17026, r17031: Move gamesetup strings to simulation/data/settings/
(#3355)
r16906: Move engine version from XmppClient.cpp
to Pyrogenesis.cpp
and use it in savegames (#3359)
r16976: Lists are sorted when clicking on the first item (not contained in the patch above, but relevant) (#3391)
r17004 Don't create empty commands.txt (#3408)
r16896 Allow pausing during visual replays
r16897, r16898 Close and free replay streams
r16902 Reliably send the ReplayFinished
message at the very end of visual replays
Thanks leper, Itms and mimo for taking the time to review and commit the individual parts!
The new patch soon to be uploaded will include almost only new files implementing the replay menu.
by , 9 years ago
Attachment: | t3258_remember_replay_directory_and_save_metadata_v1.patch added |
---|
Adds a getter in ReplayLogger
, so that we can retrieve the directory path that contains the commands.txt
file after the game started. Also adds the timestamp
, g_modsLoaded
, and engine_version
(#3359) when the game starts. This way we will be able to filter outdated / incompatible replays.
by , 9 years ago
Attachment: | t3258_rename_autostart_visualreplay_v1.patch added |
---|
Renames GameSetup::VisualReplay()
to GameSetup::AutostartVisualReplay()
.
by , 9 years ago
Attachment: | t3258_move_map_description_v1.patch added |
---|
Rebased. This should be committed separately.
by , 9 years ago
Attachment: | t3258_visual_replay_menu_v9.1.patch added |
---|
Everything needed for r17051. Updates the opening of the replayfiles, so that non-latin chars on unix are not supported anymore, but it should compile on windows on the other hand. Rebased as we could commit the changes to lobby.js
and functions_utility.js
already.
by , 9 years ago
Attachment: | t3258_visual_replay_menu_v9.2.patch added |
---|
Adds script conversions ToJSVal<time_t>
and ToJSVal<u64>
. Update by Itms.
comment:11 by , 9 years ago
9.1 compiles on linux, but not on windows. 9.2 compiles on windows, but not on linux.
Notice that 9.2 loops indefintely in VisualReplay::LoadReplayData
on windows, but not on linux.
by , 9 years ago
Attachment: | t3258_missing_script_conversions_for_windows_v1.patch added |
---|
These script conversions are required by windows. Either we need to modify this patch so that it compiles on linux too, or I need to change the code so that it doesn't need them.
by , 9 years ago
Attachment: | t3258_visual_replay_menu_v10_linux.patch added |
---|
comment:12 by , 9 years ago
As mentioned above, the prior version does an infinite loop on windows. Stan helped figuring out that removing replayStream->clear();
leads to that infinite loop instead of throwing the error message 5 lines below in case of !replayStream->good()
.
As mentioned in http://www.cplusplus.com/reference/ios/ios/clear/, this function resets the eof
- and fail
-bit.
Not sure what happens differently after that for the two operating systems yet. Couldn't test on windows yet. Above I uploaded a patch that rewrites the affected function VisualReplay::LoadReplayData
, which in theory should never reach the fileending. Works for linux.
If you want to test it on windows, you will probably need t3258_missing_script_conversions_for_windows_v1.patch
again.
Changes:
- For easier testing: added some optional
debug_printf()
calls. Enable them by settingconst bool debugParser = true;
. Please report errors on windows with that debug output and commands.txt. - Does not use
replayStream->clear();
to reset the failbit / eofbits when attempting to read at EOF. - Reduced the infinite loop protection from 10 million to 1000. If it doesn't fail there are 5 iterations on average.
- Doesn't call
m_Stream->close()
anymore: http://www.cplusplus.com/reference/fstream/ifstream/close/, butSAFE_DELETE
whenever leaving the stream.
Updates & test reports for windows will follow soon enough hopefully!
comment:14 by , 9 years ago
We can make it compile on both systems by not updating the script conversions and instead converting the affected variables explicitly in CReplayLogger::StartGame
and VisualReplay::LoadReplayData
:
< + m_ScriptInterface.SetProperty(attribs, "timestamp", (i32) std::time(nullptr)); --- > + m_ScriptInterface.SetProperty(attribs, "timestamp", std::time(nullptr)); 1577c1577 < + scriptInterface.SetProperty(replayData, "filemod_timestamp", std::to_string(fileTime)); --- > + scriptInterface.SetProperty(replayData, "filemod_timestamp", fileTime);
by , 9 years ago
Attachment: | t3258_visual_replay_menu_v11.patch added |
---|
Compiles on both windows and linux and doesn't freeze.
follow-up: 16 comment:15 by , 9 years ago
Works fine for me on Windows 10. Takes some time to load for the first time, maybe a loading screen ? else it's cool :)
comment:16 by , 9 years ago
Replying to stanislas69:
Works fine for me on Windows 10. Takes some time to load for the first time
Performance Profile:
Made some profilings with 2500 replay files.
Result: everything seems to be optimized, only HDD I/O overhead remaing. Not a mission critical problem in my opinion and the optimization can be done easily. The slow-loading occurs only the first time you load it. Instead of adding a loading screen, we should add a custom cache file to optimize.
C++ parsing: In order to display the players, map and duration, we have to load these files commands.txt files. As leper suggested, this is done the fastest way by parsing only the first line (header) and then read the file backwards to get the game duration from the last turn number. The C++ parsing of the files is optimized.
JS parsing & GUI updates: The javascript part loads in half a second with 2500 files on my system. I've measured the time spent in every function and it is evenly distributed (about 30ms per function). Could be improved maybe, but saving a fraction of a second is not really worth it now.
Disk I/O: The real underlying problem with the first load being slow is that we have to load a big amount of small files. While the parsing of the files and the GUI is optimized, we still have all those disk I/Os calls. Every file is located in a different place on the harddrive and thus the disk has to do mechanical work in that time. But everytime a file is read, it will cache that file in a small memory inside the HDD. This is the reason why on the second call it only takes 0,5s!.
You can reproduce the disk I/O calls on linux by clearing the HDD cache:
echo 3 | sudo tee /proc/sys/vm/drop_caches
The other game parts have the same problem. The HDD cache just needs to be loaded at some point. The game-data that doesn't change is zipped and uses VFS
, see wiki:Virtual_File_System. leper suggested using it. But using it is challenging, some required functions seem missing with the current implementation. More importantly this doesn't eliminate the actual problem of having disk I/O for many scattered files once.
The ideal solution to that issue is to keep a single, custom cache file (json format) that contains the data we are interested in. We would have to do the disk I/O calls only once per replay, instead of once per "replay viewing session". This means the actual loading of the files happens only once, ever.
Implementing such a cache is a low hanging fruit. Will start working on it right now, though it would be cleaner to implement and easier to review if it is added in a separate patch.
comment:17 by , 9 years ago
I think best thing is to get that patch commited, and make another one to enhance performance.
by , 9 years ago
Attachment: | t3258_visual_replay_menu_v11_PROFILING.patch added |
---|
Same patch above, but with profiling output. Notice that showing the output also consumes performance.
comment:21 by , 9 years ago
We actually made it!!!
It would not have been possible to finish this,
- if leper wouldn't have committed the patches that Itms couldn't review
- if Itms wouldn't have committed the patches that leper couldn't review
- if Itms wouldn't have continued the reviewing when leper was overworked
- if mimo wouldn't have continued the reviewing when Itms was overworked
- if Stan wouldn't have helped debugging it on windows
If any of you guys were missing, we couldn't have made it! Thanks for the endurance and going through dozens of iterations of these patches!
Here the history (28 patches consuming 180kb, introducing 2792 new lines and removing 565 existing ones):
Feb 7 Requirements analysis Feb 12 < elexis> if i wont get struck by thunderstrike i will implement this for a19 Mar 9 Alpha 18 release Apr 3 First visual replay patch Jun 6 r16727 Itms Visual Replay from command line (#9) Jun 12 First replay menu patch Jun 14 r16765 ben Update readme (#9) Jun 18 r16790 sanderd17 Fix out-of-sync in replays (#9) Jul 8 r16839 Itms Playernames in replay files (#3253) Jul 14 r16854 wraitii Fix previous commit (#3253) Jul 30 r16895 leper Add clipping to OList cells (#3353) Jul 30 r16896 leper Pausing in visual replays (#3341) Jul 30 r16897 leper Close and free replay streams Jul 30 r16898 leper Fix previous commit Jul 30 r16902 Itms Fix "replay finished" dialog (#9) Jul 30 Splitting visual replay menu patch, as it grew too big (ticket:3258#comment:8) Aug 1 r16906 leper Move engine version to Pyrogenesis.cpp (#3359) Aug 11 First version of the gamesetup cleanup (#3355) Aug 27 Splitting gamesetup cleanup, as it grew too big Aug 27 r16942 leper Enhance i18n extractor (#3355) Sep 2 r16975 Itms Move player and team number limits (#3355) Sep 2 r16976 Itms Lists are sorted when clicking on the first item (#3391) Sep 3 r16979 Itms Fix previous commit (#3355) Sep 11 r17004 leper Don't create empty commands.txt (#3408) Sep 12 r17006 Itms Move ceasefire settings (#3355) Sep 12 r17007 mimo Move starting resources (#3355) Sep 13 r17012 leper Move population capacities (#3355) Sep 13 r17017 mimo Rename replay variables (#9) Sep 13 r17019 mimo Fast forward replays and move game speeds (#3355) Sep 16 r17026 mimo Move victory conditions and fix #3209 (#3355) Sep 17 r17031 mimo Move maptypes (#3355) Sep 18 r17034 mimo Move AI settings (#3355) Sep 20 r17049 Itms Fix move-maptypes (#3355) Sep 20 r17050 mimo Rename autostart visual replay (#9) Sep 21 r17054 Itms Visual Replay Menu! (#3258) Sep 21 r17055 Itms Finish previous commit (#3258)
The next task is to fill the replay list with many gg's!
Works, but the GUI should be enriched. Adds a main menu entry for listing replays and starting or deleting them. Cleans visual replay a bit. Saves engine_version.txt. Prevents the creation of empty commands.txt.