Load Game Broken

[agentx]

The API doesn't continue loaded games. OnUpdate of a bot is called, but essential game objects like events, maps, gamestate are undefined.

After some hours of investigation it looks like the load path on the C++ side of the API needs a heavy update. After that further testing is needed, because there is no JS code available to verify proper game objects.

However, I've attached a Checker bot with some utils, showing the difference between a new game start and a loaded game:

new game:

------: Checker.OnUpdate.in
TICK: #3
------: Checker.OnUpdate.out

load game:

------: Checker.OnUpdate.in
TICK: #3
ERROR: events undefined
ERROR: gameState undefined
ERROR: territoryMap undefined
ERROR: entities undefined
ERROR: passabilityMap undefined
ERROR: playerData undefined
ERROR: terrainAnalyzer undefined
ERROR: timeElapsed undefined
ERROR: circularMap undefined
ERROR: barterPrices undefined
ERROR: passabilityClasses undefined
------: Checker.OnUpdate.out

If that doesn't make it into next Alpha, Load and Save buttons should be removed from the GUI to avoid confusing new users.

Edit:spelling

[agentx]

To make it clear, this has nothing to do with Serialization/DeSerialization. That was long enough discussed here: http://www.wildfiregames.com/forum/index.php?showtopic=18323&p=289985

[Josh]

How much does this impact normal game use? Does this only affect the AI? Are there user visible errors?

We'll need more info about the consequences of this bug if we're going to delay the A16 release.

[leper]

Replying to Josh:

How much does this impact normal game use? Does this only affect the AI? Are there user visible errors?

AIs don't work with saved games, nothing new or extraordinary. Moving.

[agentx]

AIs don't work with saved games, nothing new or extraordinary. Moving.

That applies to the existing AIs not for AIs in general. Normal game play very well includes save and load options.

My proposal is to assess the bug and if only a few lines of code had be changed, then include it in this alpha as it instantly opens door for better AIs coming with A17. That's not completely far fetched, since loaded games rely on functions already in place and tested with new games.

[leper]

Yes, but saving with AIs (be it the current or others that aren't even distributed with the game) has been broken for quite some time so it surely isn't a release blocker.

As we are in feature freeze for this Alpha, and there isn't even a working patch attached I'll push this to A17. We can still try to fix this early in the release cycle so AIs for A17 can still benefit from it. (And who is developing an AI based on releases instead of SVN?)

[agentx]

This is an engine bug, not an AI bug. Saving and loading is a feature since a lot of alphas. What is the purpose of rushing another Alpha which can't load/save? I see I18n is a great feature making more users happy. However, load/save makes ALL users happy.

[leper]

So we shouldn't release any _alpha_ just because it doesn't support X? Please leave this ticket at the milestone, it just won't happen for A16.

[agentx]

A release which doesn't support save/load is hardly an Alpha, call it an unfinished proof of concept. Is here nobody who can say what it takes to solve save/load games?

[Yves]

Did I misunderstand, or are you saying it worked with Alpha 15? What does this comment in _checker.js mean?

"tested with 0 A.D. Alpha 15, rev 14923"

Alpha 15 was r14386 and it didn't use the module pattern. Do you mean it worked with r14923 or has it never worked before?

[historic_bruno]

Replying to agentx:

A release which doesn't support save/load is hardly an Alpha, call it an unfinished proof of concept. Is here nobody who can say what it takes to solve save/load games?

Are you on the dev team? If not, I would suggest not abusing your Trac privileges, priorities are not for you to decide.

Saved games have always been broken for AIs, sometimes it has been covered up, sometimes not so well. It's a known problem, we have a ticket for it (#1089) already. If you read that and related discussions, you know my position at least.

However, I don't think it's worth spending too much time right now fixing AI saved games, until the AI API has settled closer to its final form. It's still more or less an "unfinished proof of concept" as you say. Sure, there may be an "easy" fix for your particular bug that can be done in less than a week, but the whole thing is fundamentally broken right now, and that is a months long task IMO.

[agentx]

#1089 is a 2 years old ticket for AIs not saving data. This is a ticket for the engine not supporting load game. If Alpha 8 did support continuing games, it should be quite easy to find and fix the bug.

@Yves, the rev is correct info.

[historic_bruno]

Replying to agentx:

#1089 is a 2 years old ticket for AIs not saving data. This is a ticket for the engine not supporting load game. If Alpha 8 did support continuing games, it should be quite easy to find and fix the bug.

The only errors mentioned in this ticket are AI-related, so #1089 seems relevant to me. That ticket mostly involves the engine, since a good bit of the AI implementation is in the engine. The problems can't be fixed only in C++ or only in JS.

If you're reporting a non-AI case of saved games failing, then please provide that info. As far as I know, it works fine.

[agentx]

@historic_bruno

Sure, save/load relies on users picking a capable AI. But as said removing load/save from the GUI is a honest solution to this bug.

[Josh]

Duplicate of #1089. Please move discussion to the more detailed ticket.

[Radagast]

historic_bruno, you had already almost solved it in a central way (the saved games serialisation). The only problem was that the file size is too big. Can we now somehow mitigate that?

About Continuation/ Discrepancy between new game and loading game: I will look into why the events are not reconstructed when loading games. (agentx' tests seem to show that they are undefined if a game is loaded while they are fine if a new game is started.)

I know this is low priority as saved games from different AI iterations are not compatible, yet I wish to help if I can. (and I think wraitii has already made progress towards getting rid of the template references. So that 0adsave files no longer are that big. But I have to check.)

[agentx]

@Radagast

Great, let me know if you start talking to HandleMessage, I'll make checker bot respond in a very short time frame.

[historic_bruno]

Replying to Radagast:

historic_bruno, you had already almost solved it in a central way (the saved games serialisation). The only problem was that the file size is too big. Can we now somehow mitigate that?

The approach will need re-visiting since the SpiderMonkey upgrade, some code was removed and there is a technical issue with the old approach (see #2428).

About Continuation/ Discrepancy between new game and loading game: I will look into why the events are not reconstructed when loading games.

Loading saved games isn't like starting a new game, in a lot of ways, and it's not meant to be. So I think what you really want to know, is how does the AI handle saved games? You should look at what Aegis does, since as far as I know, wraitii modified the logic to work specially for Aegis. But the whole interface should change per #1089, which is why I refer back to that ticket.

[agentx]

new game = map + 0 events + 0 commands
load game = map + x events + z commands

you won't get easier than that.

Edit: As added value, you got replay without any further ado

[Josh]

Replying to agentx:

new game = map + 0 events + 0 commands
load game = map + x events + z commands

you won't get easier than that.

Edit: As added value, you got replay without any further ado

Processing x events and z commands is impractical as it could take hours on long games.

[agentx]

Processing x events and z commands is impractical as it could take hours on long games.

The developer's overlay time warp is pretty fast.

[agentx]

Please move discussion to the more detailed ticket.

Completely inappropriate and not targeting towards a solution. The proposed ticket is stuck since years, because it ignores best practices. AI serialization is naturally a dead end, because an RTS AI can't work on state.

A simple Google search with "AI Serialization" returns around 30 hits, but "implement game save replay" 8 Million. Directly on the first page is a detailed Gamasutra article with code examples showing how to deal with input and output along the dimension of time.

​http://www.gamasutra.com/view/feature/131466/instant_replay_building_a_game_.php?print=1

The author mentions solely the issues of deterministic behaviour and floating point maths, both are already solved in 0 A.D.

So, the question is still in the room: What does it take to make 0 A.D. save and load?

[sanderd17]

Replying to agentx:

A simple Google search with "AI Serialization" returns around 30 hits, but "implement game save replay" 8 Million.

Just wondering if that second query might be a little more general?

Directly on the first page is a detailed Gamasutra article with code examples showing how to deal with input and output along the dimension of time.

​http://www.gamasutra.com/view/feature/131466/instant_replay_building_a_game_.php?print=1

The author mentions solely the issues of deterministic behaviour and floating point maths, both are already solved in 0 A.D.

So, the question is still in the room: What does it take to make 0 A.D. save and load?

May I say that 0 A.D. doesn't replay a lot faster than it plays? Near the middle of the game, almost all time is needed to calculate the new simulation state, and almost nothing goes to rendering. This means, if some player wants to load a game he left at minute 45, he has to wait some 20-30 minutes to load his game?

As such, we require the AI to be able to have a deterministic state every turn. A state that can be serialised (pretty much everything can be serialised, but the current bots seems to like self-referencing and massively big objects, so the serializing takes too long). And the serialised version of the state must be used to recreate the AI state as it was.

I first thought this issue was about something else, I thought when a new unit was killed after a loaded game f.e., the AI didn't receive any updates on that (maybe something in the AI Interface wasn't initialised correctly for saved games). But instead, you expect to see all previous messages. This is something impossible to achieve unless everyone has access to a supercomputer suddenly.

Now, either leave this ticket closed, or give us an implementation that works like you want (without serialisation and whatnot) so we can review it.

[agentx]

Now, either leave this ticket closed, or give us an implementation that works like you want (without serialisation and whatnot) so we can review it.

So, load/save bug is a wontfix. At least a statement.

[Yves]

Replying to agentx:

So, load/save bug is a wontfix. At least a statement.

Sander just said the problem will most likely have to be solved in a way which is already described by the other ticket (with serialization of the AI state). I agree on that. I've spent some time thinking about a possible solution for AI serialization in the past two weeks and have a plan how it could work now. It's quite complicated though and I'd like to experiment a bit more before describing that approach any further.

IMO serialization of the AI state is a central part of the AI interface that still needs to be figured out and it has high priority for me because it could influence other parts of the interface/API. I'd like to hear your input as an AI developer once I have something to show.

For now, please try to understand that we're all doing this in our free time and there are plenty of things to fix/improve/design etc. We often can't help new modders or developers instantly by implementing all the features they request (even though we'd like to do that and even if the features are already near the top of our own list).

[historic_bruno]

Replying to agentx:

Completely inappropriate and not targeting towards a solution. The proposed ticket is stuck since years, because it ignores best practices. AI serialization is naturally a dead end, because an RTS AI can't work on state.

In 0 A.D. (a RTS), AI is simply another component in the simulation. Every other component handles serialization (and saved games) because they were designed for that. Why can't the AI? Two reasons:

1. We only have an AI because Philip made a quick prototype years ago. Saved games weren't the focus, getting a simple AI to run in the simulation was (As I recall, saved games and AI were implemented around the same time, but saved games w/AI have always been broken),
2. The prototype was extended (a lot) over time for more complex AIs, but nobody really made the design decision on how to handle serialization. It was considered less important than providing a challenging single player mode.

Now, unfortunately, because of (2) there's no quick solution. Getting a simulation component to properly save and load in 0 A.D. is almost entirely about serialization.

If you want an AI that's not part of a simulation component, isn't state-based, and doesn't use serialization, honestly you're barking up the wrong tree. That's not fixing the current system, but writing a completely new AI system. I have to question why you would need that:

Replying to agentx:

AI serialization is naturally a dead end, because an RTS AI can't work on state.

I don't know how you're thinking of this, but maybe an analogy helps. If I put Windows into hibernate mode, everything I was doing gets saved to disk. When I want to restore, it gets loaded from disk back to memory, and I should be looking at all my same programs and documents again, running at exactly the point they left off. It wasn't necessary to save every action I had ever done on my computer to return to what I was doing, that's inherent in the state. Nothing says I couldn't have some program running that records all my actions, it would also be included in the state.

Why must AIs be different, in any genre of game?

[agentx]

Now month later I still don't see the reason for serialization. There is not a single bit of information Hannibal needs to save. I understand you've spend a lot of thoughts to make save/load working, but maybe we can work on an alternative making save/load much easier?

@historic_bruno

I agree with our analogy. It is just an AI is conceptually not part of the game instead it is a player - artificial, but still a player. And the same way you don't save a human player's plans or thinking (actually you can't) you don't need to save an AI's plans or thinking or state. An AI figures out what to do next based on the state of the game and not based on his own state. That is the only job of an AI and usually they are very good at it.

Back to the ticket: I've asked to provide a complete game state after loading and not for a load/save interface. Is it possible to do this as step one?

[historic_bruno]

Can you guarantee that your AI will behave 100% identically if it starts at some arbitrary point in a game (on loading a saved game, or rejoining multiplayer), rather than if it had run uninterrupted to that point? By identically, I mean every command it sends occurs on the same turn with the same data, every decision made in the same way. If so, then indeed your AI doesn't need serialization, all you need to do is not serialize anything and it should work fine.

One reason I can think of not to have the AI act like a player instead of a simulation component is in that case you must blindly trust that whoever is running the AI is not going to cheat. AIs can process far more actions than a human player. When the AI is part of the simulation, every player runs all the AIs and they all must have identical behavior or it breaks the synchronization. But if a single player (e.g. the host) runs the AI and the only thing other players receive are the commands, maybe the AI could tribute its resources to that player or not attack them but focus on the other players.

There may be other stronger arguments for having the AI as part of the simulation, I discussed this with Philip at some point, I don't remember everything he had to say, but it's somewhere in IRC logs.

[agentx]

Can you guarantee that your AI will behave 100% identically...

No, there is no need to, because same applies to human players as well. If I save a game and continue it a year later I won't remember what I had in mind.

What I can guarantee is given a gamestate the AIs on all multiplayer machines do the same. This relies on ALL game information is in the state.

I don't get the points about cheating/tributes, could you give a more elaborated example?

Could you link to the discussion with Philip here?

[agentx]

Is it possible that we have a different understanding of the role of the data the API holds? Like EntityCollections, MetaData and the obstruction map?