Opened 9 years ago

Closed 9 years ago

#3107 closed defect (fixed)

OOS on rejoin - mirages and visibilities of foundations

Reported by: elexis Owned by: Itms
Priority: Release Blocker Milestone: Alpha 19
Component: Core engine Keywords:
Cc: Patch:

Description

We played a 4v4 and had an out of sync. WhiteLion played as yellow, I played as green. At some point he disconnected. I think the oos came after the rejoin.

I provided WhiteLion and my oos_dump and my commands.txt.

The diff is very short. Until now leper found:

01:39 <@leper> seems like mirages and visibilities or something like that 01:40 <@leper> notice two consecutive entities with identical positions, owner, and some other values?

Attachments (2)

oos_dump_commands.txt_elexis_WhiteLion.7z (758.4 KB ) - added by elexis 9 years ago.
t3107_bugfix_a18.patch (1.7 KB ) - added by elexis 9 years ago.
Same patch as Itms committed, but for alpha 18. (Used for debugging other oos tickets)

Download all attachments as: .zip

Change History (15)

comment:1 by elexis, 9 years ago

The first differences in the dumps is indeed in the visibility count property.

Similar to the oos in #3133 we have a bunch entities missing in the end and accordingly incorrect entity IDs after that.

Last edited 9 years ago by elexis (previous) (diff)

comment:2 by elexis, 9 years ago

Besides the visibilities count and missing entities, there is no difference in the logs. The missing entities in the end are probably irrelevant.

Maybe related or identical bug as in #3151 .

comment:3 by leper, 9 years ago

Cc: Itms added

Those entities are local entities and are just there because we dump everything. Local entities are not part of the serialized simulation state (nor do (should) they have any influence on it, so the first thing to do when diffing is remove the local ents from both states.

comment:4 by Itms, 9 years ago

Cc: Itms removed
Component: UI & SimulationCore engine
Milestone: Alpha 19
Resolution: duplicate
Status: newclosed

Duplicate of #3151. Taking care of it.

comment:5 by elexis, 9 years ago

Here a diff (without the local entities):

----
--- oos_dump_elexis.txt	2015-03-17 23:24:09.000000000 +0100
+++ oos_dump_whiteLion.txt	2015-03-17 23:42:17.000000000 +0100
@@ -200033,20 +200033,20 @@
     flags: 0
     key: 15016
     x: 683.13666
     z: 687.07673
     vision: 0
-    visibilities: 16
+    visibilities: 0
     retain in fog: 1
     owner: 3
     in world: 1
     flags: 3
     key: 15017
     x: 683.13666
     z: 687.07673
     vision: 0
-    visibilities: 0
+    visibilities: 16
     retain in fog: 1
     owner: 3
     in world: 1
     flags: 1
     key: 15018
@@ -201896,11 +201896,11 @@
     flags: 1
     key: 15238
     x: 521.91749
     z: 565.77265
     vision: 0
-    visibilities: 1
+    visibilities: 0
     retain in fog: 1
     owner: 1
     in world: 1
     flags: 1
     key: 15239
@@ -201914,11 +201914,11 @@
     flags: 0
     key: 15240
     x: 521.91749
     z: 565.77265
     vision: 0
-    visibilities: 0
+    visibilities: 1
     retain in fog: 1
     owner: 1
     in world: 1
     flags: 1
     key: 15241

So the only relevant differences are that for two entities visbilities, the visbilities bitset is swapped between the entity and the mirage for the host (left) and the rejoined client (right).

15016 (visbilities 16 vs. 0) 15017 (visbilities 0 vs. 16) 15238 (visbilities 1 vs 0) 15240 (visbilities 0 vs 1)

id: 15016 template: "foundation|structures/athen_civil_centre"

id: 15017 template: "mirage|foundation|structures/athen_civil_centre"

id: 15238 template: "foundation|structures/maur_outpost"

id: 15240 template: "mirage|foundation|structures/maur_outpost"

entity: 15016, 15017 x: 683.13666 z: 687.07673

key: 15238, 15240 x: 521.91749 z: 565.77265

comment:6 by elexis, 9 years ago

Player 1 has researched the tech to increase outpost vision range:

cmd 1 {"type":"research","entity":15576,"template":"vision_outpost"}

and that entity 15240 is a mauryan outpost that belongs to Player 1 (so it's affected by that tech):

    key: 15240
    x: 521.91749
    z: 565.77265
    vision: 0
    visibilities: 0
    retain in fog: 1
    owner: 1

The visiion range tech has the following effect:

"modifications": [{"value": "Vision/Range", "multiply": 1.5}],

The unupgraded outpost range is 80 (so 120 with the upgrade).According to pythagoras, the distance between the outpost and that athen civic centre is 201,8. The outpost attack range is 55. So if those numbers can be compared, then the outpost should not be able to see nor attack the CC.

Visual replay (patch here #9) can show more events that happened to those two entities.

comment:7 by sanderd17, 9 years ago

The mauryan outpost is a foundation, so it has no vision at all on that point.

Now, it is interesting to see that both entities are foundations, and both are fogged. Visibility mask 1 means fogged for player 1, invisible for anyone else. Visibility mask 16 means fogged for player 3, and invisible for anyone else.

So we have the interesting case that it shows the fogged real entity on the host, and the fogged miraged entity on the rejoined client. At least in the first case, it's a foundation that's fogged for the owner.

Here's what I think what happens: A player places the foundation in FoW. The foundation isn't miraged (perhaps on purpose, I guess the owner might need to see if his foundation gets attacked, perhaps it's a bug). But when a player joins, the RangeManager recalculates the derived data, and maybe sends a visibilityChanged message that causes the miraging.

To test this theory, it should be reproducible with the following minimal steps.

  • The host explores a small bit of territory, and returns to the base
  • Then the host puts a foundation of a building (like an outpost) on the expored territory. But makes sure that the visibility state of that object stays unchanged (so task your units back to the base and don't construct it).
  • Then the client drops and rejoins and should get an OOS.

comment:8 by sanderd17, 9 years ago

Found an easier way to check the issue in a single-player game:

  • Open a new game, and place a foundation at the edge of your territory (don't walk near it, task your builder back to the CC ASAP).
  • Open the developer overlay, and display the selected entity state
  • Check that the template is "foundation|*" (it's probably under the chat message, so you might have to wait a bit).
  • Save the game and close the game.
  • Reload the game you just saved
  • Check the entity state again, and you'll now see that the template is "mirage|foundation|*"

comment:9 by sanderd17, 9 years ago

Milestone: Alpha 19
Resolution: duplicate
Status: closedreopened

Reopened because it's not a complete duplicate of #3151. The other ticket isn't about foundations, and doesn't have visibilities switched between the miraged and the real entity + this ticket has some valuable information now.

comment:10 by elexis, 9 years ago

There are three reproducible inconsistencies with visibilities:

(1) Allies can't see foundations in the fog of war until discovered. In general you can see the same entities as your ally, so it would be logical if he/she could see your foundations too. Then you can place a dock or a CC and your ally can build it. Currently the foundation becomes visible (and repairable) if its in the line of sight of the player or an ally.

(2) You can see the actual entities instead of the mirages if you place a foundation in the fog of war . Side effect: you can delete foundations placed in the fog of war (because the entity is visible), but you cannot delete foundations in the fog of war that were discovered (mirage).

Reproduce: (a) Open the developers overlay. (b) Place a barracks in the fog of war. Notice you can delete it. (c) Move a unit so that the foundation is in the line of sight. Don't build, but move the unit back so that the foundation lies in the fog of war again. (d) Notice you can't delete it anymore.

(3) Entity and mirage disappear if an enemy destroys a foundation in the fog of war. The foundation shouldn't even be visible to the enemy until commited.

Last edited 9 years ago by elexis (previous) (diff)

in reply to:  7 comment:11 by elexis, 9 years ago

sanderd17's recipe for reproducing the bug works. If you place a barracks in the fog of war, stop your unit and rejoin with the other player, an oos occurs.

Using attachment:t3255_use_timestamp_and_pid_for_oosdump_filename.patch:ticket:3255 you can diff the two oos dumps and see that it has the same result as the diff originally uploaded:

--- oos_dump_143286275730991.txt	2015-05-29 03:25:57.230451000 +0200
+++ oos_dump_143286275731044.txt	2015-05-29 03:25:57.226451000 +0200
@@ -8611 +8611 @@
-    visibilities: 1
+    visibilities: 0
@@ -8621 +8621 @@
-    visibilities: 0
+    visibilities: 1

Related: #2710

Last edited 9 years ago by elexis (previous) (diff)

comment:12 by elexis, 9 years ago

Summary: OOS on rejoin - mirages and visibilitiesOOS on rejoin - mirages and visibilities of foundations

comment:13 by Itms, 9 years ago

Owner: set to Itms
Resolution: fixed
Status: reopenedclosed

In 16857:

Change the handling of modified entities in the visibility update.

The game has to deal with situations such as: the visibility of an entity changes, a mirage is created for it -> the mirage visibility is updated -> the entity visibility is updated back.
All of this process now happens in the same turn, and all updates are guaranteed to be performed. This fixes a source of serialization errors and rejoin OOSes.

Fixes #3107

by elexis, 9 years ago

Attachment: t3107_bugfix_a18.patch added

Same patch as Itms committed, but for alpha 18. (Used for debugging other oos tickets)

Note: See TracTickets for help on using tickets.