[Git] [Step 1] Update autobuild utility for git workflow

[historic_bruno]

Currently the Windows autobuilder works with our SVN repository (see BuildServerSetup). As we're transitioning to git, we need it to work with a git repo instead.

• support multiple git branches, both the "master" and various feature branches, so e.g. artists can easily test new features
• commit a file for the relevant branch, that can be tested by an updater utility, and contains paths and checksums of each file that was built
• the built files will be uploaded to a web server

See also #1814 and #1816.

[historic_bruno]

How will we clean up old autobuild files, e.g. feature branches that have been merged into the development branch and removed?

If I'm correctly understanding the proposed design, it will be like this: http://foo.wildfiregames.com/autobuild/master/

Then someone creates a def-lighting feature branch and autobuilds it: http://foo.wildfiregames.com/autobuild/def-lighting/

We approve it and it gets merged into master, what happens to the def-lighting directory? Should the autobuilder itself be responsible for checking the branches and pruning old ones?

[Kieran P]

This is what I had in my head: The auto builder needs to know which branches it can build, so it needs to either maintain a git repo and get the branch list from that OR use the github API to get the branch list. Either way, for it to see that a branch is gone, we'd trigger an update of the autobuilder branch list. When it sees a branch is gone, it disappears from the list and removed the folder for that branch.

[historic_bruno]

See also BuildAndDeploymentEnvironment#Futuredesign, I updated it to our current plans.

[fabio]

Why the "trunk" directory under master branch and not directly keeping its subdirectories?

[historic_bruno]

Replying to fabio:

Why under master branch the "trunk" directory and not directly its subdirectories?

Good question :) I corrected that and improved the formatting a bit.

[fabio]

Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad.

I already suggested this here: ​http://trac.wildfiregames.com/ticket/304#comment:4

[Jonathan Waller]

Replying to fabio:

Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad.

I already suggested this here: ​http://trac.wildfiregames.com/ticket/304#comment:4

We want to try and keep the process as simple as possible for the main development team. Having to update two repos goes counter to this and will almost inevitably lead to someone wasting time because they forgot to update one and now hit an issue. A small amount of extra downloading for people wishing to use the engine isn't so bad. Also many mods will want to use our javascript which is part of the public mod anyway.

[historic_bruno]

And anyway there's not a clear division between engine/mod/data currently, so until (if) we make that clearer, it's not worth worrying about separating them in different repos. As-is, source/ wouldn't be usable without binaries/data/.

[historic_bruno]

Latest plan is to have "our" own autobuild server running Jenkins, as outlined here: JenkinsSetup

The autobuilt files will be stored in an SVN repo and retrieved by the update script according to whatever git branch the user has checked out. It would be nice if all dependencies also had MSVC projects that could be built automatically this way, instead of the current approach of building them individually with a mix of compilers, then forgetting how it's done. But that's not needed initially.

(Apart from git, we want to upgrade the autobuild compiler to VC++ 2010 or 2012, but Philip says it's as much work to do that with the current system as it would be to implement the new one.)

[mimo]

moved to backlog as no progress since ages