Opened 4 years ago

Last modified 14 months ago

#1819 new enhancement

[Git] [Step 1] Update autobuild utility for git workflow

Reported by: historic_bruno Owned by:
Priority: Must Have Milestone: Backlog
Component: Build & Packages Keywords:
Cc: k776

Description

Currently the Windows autobuilder works with our SVN repository (see BuildServerSetup). As we're transitioning to git, we need it to work with a git repo instead.

  • support multiple git branches, both the "master" and various feature branches, so e.g. artists can easily test new features
  • commit a file for the relevant branch, that can be tested by an updater utility, and contains paths and checksums of each file that was built
  • the built files will be uploaded to a web server

See also #1814 and #1816.

Change History (20)

comment:1 Changed 4 years ago by historic_bruno

How will we clean up old autobuild files, e.g. feature branches that have been merged into the development branch and removed?

If I'm correctly understanding the proposed design, it will be like this:
http://foo.wildfiregames.com/autobuild/master/

Then someone creates a def-lighting feature branch and autobuilds it:
http://foo.wildfiregames.com/autobuild/def-lighting/

We approve it and it gets merged into master, what happens to the def-lighting directory? Should the autobuilder itself be responsible for checking the branches and pruning old ones?

comment:2 Changed 4 years ago by k776

This is what I had in my head: The auto builder needs to know which branches it can build, so it needs to either maintain a git repo and get the branch list from that OR use the github API to get the branch list. Either way, for it to see that a branch is gone, we'd trigger an update of the autobuilder branch list. When it sees a branch is gone, it disappears from the list and removed the folder for that branch.

comment:3 Changed 4 years ago by k776

  • Summary changed from Update autobuild utility for git workflow to [Git] [Step 1] Update autobuild utility for git workflow

comment:4 Changed 4 years ago by historic_bruno

See also BuildAndDeploymentEnvironment#Futuredesign, I updated it to our current plans.

comment:5 follow-up: Changed 4 years ago by fabio

Why under master branch the "trunk" directory and not directly its subdirectories?

Last edited 4 years ago by fabio (previous) (diff)

comment:6 in reply to: ↑ 5 Changed 4 years ago by historic_bruno

Replying to fabio:

Why under master branch the "trunk" directory and not directly its subdirectories?

Good question :) I corrected that and improved the formatting a bit.

comment:7 follow-up: Changed 4 years ago by fabio

Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad.

I already suggested this here: http://trac.wildfiregames.com/ticket/304#comment:4

comment:8 in reply to: ↑ 7 Changed 4 years ago by quantumstate

Replying to fabio:

Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad.

I already suggested this here: http://trac.wildfiregames.com/ticket/304#comment:4

We want to try and keep the process as simple as possible for the main development team. Having to update two repos goes counter to this and will almost inevitably lead to someone wasting time because they forgot to update one and now hit an issue. A small amount of extra downloading for people wishing to use the engine isn't so bad. Also many mods will want to use our javascript which is part of the public mod anyway.

comment:9 Changed 4 years ago by historic_bruno

And anyway there's not a clear division between engine/mod/data currently, so until (if) we make that clearer, it's not worth worrying about separating them in different repos. As-is, source/ wouldn't be usable without binaries/data/.

comment:10 Changed 4 years ago by k776

  • Milestone changed from Alpha 13 to Alpha 14

comment:11 Changed 4 years ago by historic_bruno

Latest plan is to have "our" own autobuild server running Jenkins, as outlined here: JenkinsSetup

The autobuilt files will be stored in an SVN repo and retrieved by the update script according to whatever git branch the user has checked out. It would be nice if all dependencies also had MSVC projects that could be built automatically this way, instead of the current approach of building them individually with a mix of compilers, then forgetting how it's done. But that's not needed initially.

(Apart from git, we want to upgrade the autobuild compiler to VC++ 2010 or 2012, but Philip says it's as much work to do that with the current system as it would be to implement the new one.)

comment:12 Changed 4 years ago by k776

  • Milestone changed from Alpha 14 to Alpha 15

comment:13 Changed 3 years ago by leper

  • Milestone changed from Alpha 15 to Alpha 16

comment:14 Changed 3 years ago by leper

  • Milestone changed from Alpha 16 to Alpha 17

comment:15 Changed 3 years ago by historic_bruno

  • Owner Philip deleted

comment:16 Changed 3 years ago by leper

  • Milestone changed from Alpha 17 to Alpha 18

comment:17 Changed 2 years ago by Itms

  • Milestone changed from Alpha 18 to Alpha 19

comment:18 Changed 21 months ago by mimo

  • Milestone changed from Alpha 19 to Alpha 20
  • Priority changed from Release Blocker to Must Have

comment:19 Changed 14 months ago by mimo

moved to backlog as no progress since ages

comment:20 Changed 14 months ago by mimo

  • Milestone changed from Alpha 20 to Backlog
Note: See TracTickets for help on using tickets.