Opened 5 years ago

Last modified 18 months ago

#1819 new enhancement

[Git] [Step 1] Update autobuild utility for git workflow

Reported by: Ben Brian Owned by:
Priority: Must Have Milestone: Backlog
Component: Build & Packages Keywords:
Cc: Kieran P Patch:

Description

Currently the Windows autobuilder works with our SVN repository (see BuildServerSetup). As we're transitioning to git, we need it to work with a git repo instead.

  • support multiple git branches, both the "master" and various feature branches, so e.g. artists can easily test new features
  • commit a file for the relevant branch, that can be tested by an updater utility, and contains paths and checksums of each file that was built
  • the built files will be uploaded to a web server

See also #1814 and #1816.

Change History (20)

comment:1 Changed 5 years ago by Ben Brian

How will we clean up old autobuild files, e.g. feature branches that have been merged into the development branch and removed?

If I'm correctly understanding the proposed design, it will be like this: http://foo.wildfiregames.com/autobuild/master/

Then someone creates a def-lighting feature branch and autobuilds it: http://foo.wildfiregames.com/autobuild/def-lighting/

We approve it and it gets merged into master, what happens to the def-lighting directory? Should the autobuilder itself be responsible for checking the branches and pruning old ones?

comment:2 Changed 5 years ago by Kieran P

This is what I had in my head: The auto builder needs to know which branches it can build, so it needs to either maintain a git repo and get the branch list from that OR use the github API to get the branch list. Either way, for it to see that a branch is gone, we'd trigger an update of the autobuilder branch list. When it sees a branch is gone, it disappears from the list and removed the folder for that branch.

comment:3 Changed 5 years ago by Kieran P

Summary: Update autobuild utility for git workflow[Git] [Step 1] Update autobuild utility for git workflow

comment:4 Changed 4 years ago by Ben Brian

See also BuildAndDeploymentEnvironment#Futuredesign, I updated it to our current plans.

comment:5 Changed 4 years ago by fabio

Why under master branch the "trunk" directory and not directly its subdirectories?

Last edited 4 years ago by fabio (previous) (diff)

comment:6 in reply to:  5 Changed 4 years ago by Ben Brian

Replying to fabio:

Why under master branch the "trunk" directory and not directly its subdirectories?

Good question :) I corrected that and improved the formatting a bit.

comment:7 Changed 4 years ago by fabio

Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad.

I already suggested this here: http://trac.wildfiregames.com/ticket/304#comment:4

comment:8 in reply to:  7 Changed 4 years ago by Jonathan Waller

Replying to fabio:

Did you consider having two different repositories for the engine (pyrogenesis) and the game data (0ad)? This should possibly make things easier for who want to use the engine for a new game (at least can follow pyrogenesis git without getting all unrelated art assets). The generated packages should also be renamed from 0ad to pyrogenesis and from 0ad-data to 0ad.

I already suggested this here: http://trac.wildfiregames.com/ticket/304#comment:4

We want to try and keep the process as simple as possible for the main development team. Having to update two repos goes counter to this and will almost inevitably lead to someone wasting time because they forgot to update one and now hit an issue. A small amount of extra downloading for people wishing to use the engine isn't so bad. Also many mods will want to use our javascript which is part of the public mod anyway.

comment:9 Changed 4 years ago by Ben Brian

And anyway there's not a clear division between engine/mod/data currently, so until (if) we make that clearer, it's not worth worrying about separating them in different repos. As-is, source/ wouldn't be usable without binaries/data/.

comment:10 Changed 4 years ago by Kieran P

Milestone: Alpha 13Alpha 14

comment:11 Changed 4 years ago by Ben Brian

Latest plan is to have "our" own autobuild server running Jenkins, as outlined here: JenkinsSetup

The autobuilt files will be stored in an SVN repo and retrieved by the update script according to whatever git branch the user has checked out. It would be nice if all dependencies also had MSVC projects that could be built automatically this way, instead of the current approach of building them individually with a mix of compilers, then forgetting how it's done. But that's not needed initially.

(Apart from git, we want to upgrade the autobuild compiler to VC++ 2010 or 2012, but Philip says it's as much work to do that with the current system as it would be to implement the new one.)

comment:12 Changed 4 years ago by Kieran P

Milestone: Alpha 14Alpha 15

comment:13 Changed 4 years ago by leper

Milestone: Alpha 15Alpha 16

comment:14 Changed 3 years ago by leper

Milestone: Alpha 16Alpha 17

comment:15 Changed 3 years ago by Ben Brian

Owner: Philip Taylor deleted

comment:16 Changed 3 years ago by leper

Milestone: Alpha 17Alpha 18

comment:17 Changed 2 years ago by Itms

Milestone: Alpha 18Alpha 19

comment:18 Changed 2 years ago by mimo

Milestone: Alpha 19Alpha 20
Priority: Release BlockerMust Have

comment:19 Changed 18 months ago by mimo

moved to backlog as no progress since ages

comment:20 Changed 18 months ago by mimo

Milestone: Alpha 20Backlog
Note: See TracTickets for help on using tickets.