(Related to ticket #224.)

The current plan (subject to change based on any new information or experience):

• The server runs on ​Amazon EC2.
• The server has Windows Server 2003, VC++ 2008, SVN, Perl, and nothing else installed.
• The server has a persistent ​EBS disk with the SVN checkout.
• On startup, the server runs a script. The script updates SVN, builds, saves the log output somewhere, and if it was successful then it commits the new binaries.
• Some other server runs some kind of script that provides an authenticated web interface to (re)start the build server, and terminates it after some period of time when it's done.

Virtual machine image installation

The software versions listed below are the latest when I set this up - more recent versions ought to work fine too.

• Create a 2GB EBS disk in us-east-1c. (...or any other availability zone, but it needs to match the instance's zone, and I can only make Windows instances in us-east-1c.)
• Start with basic Win2003 image (ami-0456b26d, ec2-public-windows-images/Server2003r2-i386-Win-v1.03.manifest.xml), on a Small Instance in us-east-1c.
• Change the Administrator password, and write it down somewhere.
• Install ​VC++ 2008 Express SP1. (Skip the Silverlight and SQL Server options). Register it.
• Install ​command-line SVN 1.5.5.
• Install ​ActivePerl.
• Install ​Net::Amazon::EC2, ​Amazon::S3, DateTime::Format::ISO8601 using the Perl Package Manager.
• Download ​SDelete, extract into c:\bin.
• Download ​Sync, extract into c:\bin.
• Create the registry key HKEY_USERS\S-1-5-18\Software\Sysinternals\Sync, set DWORD EulaAccepted = 1. (Yay.)
• Attach the EBS disk.
• If the disk hasn't been set up already: Administrative Tools -> Computer Management -> Disk Management -> Disk 2 -> Initialize Disk, New Partition -> drive letter E.
• svn co http://svn.wildfiregames.com/svn/ps/trunk/ e:\svn
• mkdir c:\0ad\autobuild\
• copy e:\svn\source\tools\autobuild2\startup.pl c:\0ad\autobuild\
• copy e:\svn\source\tools\autobuild2\aws.conf.example c:\0ad\autobuild\aws.conf
• Set up c:\0ad\autobuild\aws.conf:
  • aws_access_key_id, aws_secret_access_key are from your Amazon account.
  • ebs_volume_id is from the EBS volume you created.
  • svn_username, svn_password are the SVN login details.
• copy e:\svn\build\autobuild2\startupservice.exe c:\0ad\autobuild\
• sc create Autobuilder binPath= c:\0ad\autobuild\startupservice.exe start= auto depend= Ec2Config
• Start -> Programs -> Ec2Service Setting -> disable "Set Computer Name". (That feature changes the computer name then reboots, which is annoying.)
• Bundle the instance:
  • To save some space, delete c:\windows\$*Uninstall*$, c:\windows\temp\*.
  • Defragment drive C.
  • c:\bin\sdelete -c c:
  • Bundle into s3://wfg-autobuild-images/vc2008.
  • Register the bundle as an AMI.
• Set up manage.conf on the build-runner server:
  • image_id is from the bundled AMI.

Build script

The initial part (stored on the image on S3) does something like:

• ec2-attach-volume to attach the EBS disk.
• svn up on the EBS disk.
• Run the rest of the build script from the SVN directory.

The rest of the build script does:

• (TODO: is it more efficient if all the source is copied onto the local disk before compiling?)
• premake --target vs2008 --outpath e:\build
• (TODO: build Collada, Atlas too?)
• vcbuild /time /showenv /useenv /M4 /r /logfile:buildlog.txt pyrogenesis.sln "Release|Win32"
• (TODO: have to think about tests - should they be compiled? should they be run? what should happen if they fail?)
• Upload build logs to a public S3 bucket (because that's an easy web-accessible hosting service).
• svn commit -m 'Automated build.'
• Safely ​unmount the EBS disk.
• Call ​TerminateInstances to stop the machine.

Build runner

• Runs on same server as SVN.
• Uses SVN auth info to only allow registered developers to trigger builds (because they cost money).
• The server also runs a cron job every 5 minutes, that checks if a build was started more than 50 minutes ago, and calls TerminateInstances if so. (That should be enough time for a legitimate build to end, and prevents a runaway machine costing lots of money.)
• The web interface also provides an admin-only 'down for maintenance' mode that prevents any builds and pauses that cron job, so I can manually run the build server without interference.

EC2

Handy tools: ​ElasticFox, ​s3://.

Alternatives to EC2

Currently (December 2008) the only suitable alternatives to EC2 I've seen are ​FlexiScale and ​GoGrid. Since all the services provide a plain Windows environment, it should be fairly easy to transition between them if necessary.

EC2 appears to offer the best value. Comparing the lower-end offerings from each:

Service	RAM (GB)	CPU (EC2 units)	Machine per hour	Storage per GB-month	Total per month
EC2	1.7	1	$0.125	$0.15	$4.55
EC2	1.7	5	$0.30	$0.15	$9.88
FlexiScale	0.5	0.5 (?)	£0.08	£0.30	£3.93
FlexiScale	0.5	1 (?)	£0.11	£0.30	£4.85
FlexiScale	1.0	0.5 (?)	£0.13	£0.30	£5.45
GoGrid	0.5	0.125 (?)	$0.095	$0	$2.90
GoGrid	1.0	0.25 (?)	$0.190	$0	$5.78

(This assumes 5GB of storage, and 1 hour usage per day.)

(£1 UKP = $1.50 USD right now. Tax (15%) is not included for EC2; unsure about others.)

Regarding CPU: "One EC2 Compute Unit provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor. This is also the equivalent to an early-2006 1.7 GHz Xeon processor". FlexiScale ​says "I would expect a typical 2gb virtual server with us to get more CPU resources compared to Amazon", implying that their "2 CPU" offering is roughly comparable to 1 EC2 Compute Unit. GoGrid ​says "we will always maintain a ratio of 1 Xeon core (equivalent to a P4 2.0 chip) to 4 GB of RAM across our entire grid", which sounds like about 1 EC2 Compute Unit per 4GB.

Builds take somewhere around 20 minutes on 1 EC2 Compute Unit, which is barely adequate. Anything less than that is rubbish.

Turns out that GoGrid actually charges even when the machine is switched off, since you pay for reserved resources, so that's not acceptable. But they claim they're looking into changing that, in which case they'd be more suitable.