Invisible formation bug

[causative]

Units can get caught in an "invisible formation" that cannot be disabled. The units appear to have no formation according to the formation buttons, but they behave as if in a formation, and inspecting the simulation state, a formation exists. This can result in all units in the army becoming permanently uncontrollable, effectively meaning the entire army is lost.

While playing a multiplayer game, I told my army of ~100 slingers, Cunobelin, and some healers to organize into several "Forced March" formations and walk each formation towards the enemy. Some of the formations had a traffic jam in a forest. I issued a couple more move orders to try to resolve the jam. At this time, two invisible formations arose, and the units milled about stuck in one place. Nothing I did would move them: telling them to move, telling them to halt, telling them to have no formation, telling them to take on a different formation, ordering only a few of them at a time - none of these things had much effect.

This bug is difficult to cause intentionally. It happens reliably in the replay, however. By turn 3438 (28:39) there was a traffic jam, and I issued a command for most of the units to attack-walk. On turn 3439, there appeared formation with entity ID 9309; this is the first of the invisible formations. On turn 3440, I issued a second command for the same units to attack-walk. On turn 3441, there appeared a formation with entity ID 9315; this is the other invisible formation.

Below are turn-by-turn entity IDs of active formations I gathered from using Engine.DumpSimState().

turn 3437,3438:
    key: "formations/column_closed"
    length: 3
    value: 9291
    value: 9292
    value: 9294

(first attack-walk command issued on turn 3438)

turn 3439:
    key: "formations/column_closed"
    length: 4
    value: 9309
    value: 9310
    value: 9311
    value: 9312

turn 3440:
    key: "formations/column_closed"
    length: 1
    value: 9309

(second attack-walk command issued on turn 3440)

turn 3441:
    key: "formations/column_closed"
    length: 6
    value: 9309
    value: 9315
    value: 9316
    value: 9317
    value: 9318
    value: 9319

turn 3442:
    key: "formations/column_closed"
    length: 2
    value: 9309
    value: 9315

You can switch to player 1 in the replay or a saved game from the replay, by using Engine.GetGUIObjectByName("devCommands").hidden = false; to open the dev overlay, and then using that to change perspective to causative. You can see that there's nothing the player can do to get the units unstuck. Formation 9309 affects a small handful of units that you can see by taking control on turn 3439, and 9315 affects the rest. The attack-move commands seem somehow related, because 9309 appeared a turn after the first one, and 9315 appeared a turn after the second one.

[elexis]

This is the same bug I complained about in #4153, though that one replay/gif also contains unresolvable pathfinding issues.

If I recall correctly, there is a formation entity and the entities are assigned to it, maybe in multiple places and one of them got lost during Transform, Ownership, Upgrade, (promotion, tech, capturing ...), or ungarrisoning or whatever. I dont know the code, temple and mimo do more.

The good news is that we now have a recent replay with that bug and I have a patch to start a replay from a certian savegame. So we could skip the first 9000 turns for the investigation.

If we can link the "can't be controlled" property of the entities to a wrong simulation state, then we can track which call changed it to the wrong value, until we find the sourceo f the bug.

[elexis]

Start a visual replay from commandline after loading a savestate from command line, (a21 patch that should still apply)

[causative]

turn 3439

[causative]

turn 3437 IIRC (before the bug appears)

[temple]

In r21006 we remove units from their old formation, maybe that's part of what's happening here, hard to say.

[Imarok]

Move tickets to Simulation as UI & Simulation got some sub components.

[wraitii]

I can't reproduce this, nor the variant at #4153.

I tried garrisoning a few units, promoting a formation unit while walking, while idle. Switching formations.

Given there's been various fixes for formations lately (including ​Phab:rP23236 which sounds related), I'll assume this is indeed fixed in A24.

Please reopen if you find out otherwise.

[elexis]

(If the ticket is closed based on the grounds that we can't reproduce it (we didn't try hard I guess and there werent svn games involving formations), then one can close it as "needs info". causative mentioned he failed to reproduce it intentionally, so the fact that it couldn't be reproduced quickly can also mean just that. If there is a specific a24 commit in mind that could have fixed this, one can look at that commit and argue that the lack of the behavior change in this commit could have caused the bug for instance and in the strongest case that bugfix could even indicate how to reproduce the bug.)