[[TOC]]

== Summary ==
Parts of our build and deployment environment are currently being changed.
The main goal behind this is being able to easily create deployable packages/bundles for OSX.
This Page should be a work in progress and its purpose is to document the overall design and the different tasks required to implement it.
The idea is that people involved in these tasks discuss all topics related to that and document the decisions here.
Once we all have the same idea about the solution we aim for, we can assign the different tasks to different people and complete them more efficiently.

A good place for discussions is [http://www.wildfiregames.com/forum/index.php?showtopic=15511 this thread] in the forums.


----
= Overview current design =

== update-workspaces.sh / update-workspaces.bat: ==
The sh-script on Linux/*nix/OSX runs the compilation of some external dependencies and also premake.
On Windows the batchfile only executes premake because most of the dependencies are precompiled.

For release packaging, `update-workspaces.sh` can be used to set compile-time, absolute paths used by the game (to override defaults):
 * `--bindir=/path/to/bin` sets the location of the pyrogenesis executable (eg. /usr/bin or /opt/0ad/system). Defines the `INSTALLED_BINDIR` preprocessor directive.
 * `--libdir=/path/to/lib` sets the location of the bundled libraries (eg. /usr/lib). Defines the `INSTALLED_LIBDIR` preprocessor directive.
 * `--datadir=/path/to/data` sets the location of the game's data (e.g. /usr/share/0ad/data, /usr/local/share/0ad/data, or /opt/0ad/data). Defines the `INSTALLED_DATADIR` preprocessor directive.

== Premake ==
Premake creates our workspaces (also called solutions or projects) for the different platforms.
Currently we support GCC/make, !VisualStudio 2013, Xcode3, Xcode4, Codeblocks.

== Workspaces ==
The workspaces are used for building the code on different platform and also for development with different programs (IDE's).

== Deployment ==
For (alpha-)releases we make a special deployment for various reasons. The main reason is that we want to make it easier and more convenient for the user.
Everything is precompiled and distributed in appropriate containers for the different operating systems and platforms (e.g. .deb files for debian based linux systems and .exe files for windows etc.).


----
= Future design =

== Overview ==
Some basic design decisions
  * We have decided to migrate from SVN to Git (see #1819, #1814, #1816)
  * In the first place we stick to Premake. If someone offers a working solution with other tools like CMake, we are open to change that decision.
  * Sticking to Premake means we can't manually create workspaces
  * We remove most of the Windows binaries from the repository (starting with autobuilt exes/dlls in `binaries/system`)
  * Libraries are split into separate directories per operating system and architecture
  * Script / updater utility checks if the libraries are present and up to date to avoid problems with outdated libraries
  * The build process continues the same as before
  * A script is included to automatically build OS X bundles

== Graphical overview of interaction between different components (future-version) ==
[[Image(http://trac.wildfiregames.com/attachment/wiki/BuildAndDeploymentEnvironment/future-build-environment-overview.png?format=raw)]]

== The repository structure ==

Git repo:
{{{
master
|
|-- binaries
|   `-- system
|       `-- updater.exe / updater.bat
|
|-- build
|   |-- premake
|   `-- workspaces
|       |-- build-osx-bundle.sh
|       `-- update-workspaces.sh
|
|-- libraries
|   |-- osx
|   |   `-- build-osx-libs.sh
|   |
|   |-- source
|   |   |-- Lib A
|   |   |   |-- include
|   |   |   |-- libs
|   |   |   `-- src
|   |   `-- Lib B
|   |       |-- include
|   |       |-- libs
|   |       |-- src
|   |       `-- build.sh
|   |
|   `-- win32
|       |-- Lib C
|       |   |-- include
|       |   `-- libs
|       `-- Lib D
|           |-- include
|           `-- libs
|
`-- source
}}}

Web server storing pre-built Windows binaries:
{{{
autobuild
|
|-- master
|   `-- binaries
|       `-- system
|           |-- AtlasUI.dll
|           |-- Collada.dll
|           |-- pyrogenesis.exe
|           `-- pyrogenesis.pdb
|
`-- feature-branch
    `-- binaries
        `-- system
            |-- AtlasUI.dll
            |-- Collada.dll
            |-- pyrogenesis.exe
            `-- pyrogenesis.pdb

libraries
|
`-- win32
    |-- Lib A
    |   |-- include
    |   `-- libs
    |-- Lib B
    |   |-- include
    |   `-- libs
    |-- Lib C
    |   |-- include
    |   `-- libs
    `-- Lib D
        |-- include
        `-- libs
}}}

== Checking out the code ==
Developers will check out the code using Git instead of SVN.

== Windows build ==
Windows developers will first run `binaries\system\updater.exe` to get the latest pre-built win32 binaries. This will keep them updated for whichever branch they are testing, or they can build the game themselves.

== OS X build ==

OS X developers have the choice of building a loose binary or a bundle, with slightly different steps and requirements.

See BuildInstructions#OSX

== Linux !AppImage ==

You can use these steps to create an [https://appimage.org/ AppImage]:

Build 0ad. Using the 0ad stable distribution archives is recommended.

{{{
./clean-workspaces.sh
./update-workspaces.sh -j$(nproc) \
 && make config=release -C gcc -j$(nproc)
}}}

Copy the content in the code block below and save it to a text file (to be run as script).
READ through the comments before running the script. It will require minor edits.

{{{
#!/bin/sh
# install.sh: Installs files to prepare for building an AppImage
# Copyright 2022 The 0ad project
# 
set -ev # exit on error, be verbose

# VERSION= Set the version of the appimage (e.g
# ARCH = probably not needed, usually auto-detected

# Some other variables can be used but I couldn't find the docs for them.
# They're shown in the code here:
# https://github.com/AppImage/AppImageKit/blob/master/src/appimagetool.c

# Get a copy of 'linuxdeploy' (you can use `wget` to get the desired release/arch). 
# https://github.com/linuxdeploy/linuxdeploy/releases

# NOTE: Instead of using 'linuxdeploy', you might want to use 'appimagetool' which is a
# lower-level tool that provides other options, such as signing.
# https://github.com/AppImage/AppImageKit#appimagetool-usage

# In a docker container, you'll get an error if trying to run an appimage.
# 
# dlopen(): error loading libfuse.so.2
# AppImages require FUSE to run. 
#
# You can enter `./linuxdeploy-x86_64.AppImage --appimage-extract` to
# extract the image. It will extract to a directory named 'squashfs-root'.
# You can then execute linuxdeploy and all required arguments by using
# <path-to>/squashfs-root/AppRun [args]

if [ -z ${ARCH} ]; then
  echo "ARCH must be set"
  exit 1
fi

wget -c "https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-${ARCH}.AppImage"
chmod +x linuxdeploy-${ARCH}.AppImage

# The linuxdeploy gtk plugin is required for Atlas to work
wget -c "https://raw.githubusercontent.com/linuxdeploy/linuxdeploy-plugin-gtk/master/linuxdeploy-plugin-gtk.sh"
chmod +x linuxdeploy-plugin-gtk.sh

# 'patchelf' is required
# '-e' is set so the script will exit with an error message if this fails
patchelf --version

#ABS_PATH_WORK_DIR=
#ABS_PATH_SRC_ROOT=/0ad

if [ ! -d "${ABS_PATH_WORK_DIR}" ]; then
  echo "The work dir must be an absolute path to an existing directory."
  exit 1
fi

if [ ! -r "${ABS_PATH_SRC_ROOT}/source/main.cpp" ]; then
  echo "set the source root!"
  exit 1
fi

APPDIR=${ABS_PATH_WORK_DIR}/AppDir

cd ${ABS_PATH_SRC_ROOT}

install -s binaries/system/pyrogenesis -Dt ${APPDIR}/usr/bin
install -s binaries/system/ActorEditor -Dt ${APPDIR}/usr/bin

cd ${APPDIR}/usr/bin
ln -s pyrogenesis 0ad

for lib in libmozjs78-ps-release.so \
        libnvcore.so    \
        libnvimage.so   \
        libnvmath.so    \
        libnvtt.so
do
  patchelf --set-rpath $lib:${ABS_PATH_SRC_ROOT}/binaries/system pyrogenesis
done
patchelf --set-rpath libthai.so.0:$APPDIR/usr/lib ActorEditor
patchelf --set-rpath libAtlasUI.so:${ABS_PATH_SRC_ROOT}/binaries/system ActorEditor

# Note that binaries/system{libmoz*.so, libnv*.so, libAtlasUI.so} will be copied into
# the ${APPDIR} folder automatically when linuxdeploy is run below.

cd ${ABS_PATH_SRC_ROOT}

install binaries/system/libCollada.so -Dt ${APPDIR}/usr/lib
install build/resources/0ad.appdata.xml -Dt ${APPDIR}/usr/share/appdata
install build/resources/0ad.desktop -Dt ${APPDIR}/usr/share/applications
install build/resources/0ad.png -Dt ${APPDIR}/usr/share/pixmaps

mkdir -p ${APPDIR}/usr/data/config
cp -a binaries/data/config/default.cfg ${APPDIR}/usr/data/config

cp -a binaries/data/l10n ${APPDIR}/usr/data
cp -a binaries/data/tools ${APPDIR}/usr/data # for Atlas

# IMPORTANT: If you're creating the image from a distribution archive, this should copy
# 'binaries/data/mods/mod/mod.zip', 'binaries/data/mods/public/public.zip', and
# 'binaries/data/mods/public/mod.json'
# If you're using the svn/git version, you'll need to use the pyrogenesis
# pyromod archive builder. It's recommended to create the two zip files
# in a directory outside of the ${APPDIR} folder (in case you need to remove
# the ${APPDIR} folder later) and then copy them to ${APPDIR}/usr/data
# see https://trac.wildfiregames.com/wiki/Modding_Guide#Distributingyourmods
#
cp -a binaries/data/mods ${APPDIR}/usr/data

# Remove any symlinks to mods that may be in binaries/data/mods (this will cause 0ad to crash
# when the appimage is run). If you're creating the image from a clean 0ad archive, then
# there won't be any symlinks of course

# Create the image
cd ${ABS_PATH_WORK_DIR}

DEPLOY_GTK_VERSION=3 # Variable used by gtk plugin
./linuxdeploy-${ARCH}.AppImage -d ${APPDIR}/usr/share/applications/0ad.desktop \
  --icon-file=${APPDIR}/usr/share/pixmaps/0ad.png \
  --icon-filename=0ad \
  --executable ${APPDIR}/usr/bin/pyrogenesis \
  --library=/usr/lib/x86_64-linux-gnu/libthai.so.0 \ # not installed Gentoo
  --appdir ${APPDIR} \
  --output appimage \
  --plugin gtk
}}}

Other Notes:

To test any changes you want to make to the appimage, or for debugging, you can make changes inside the '${APPDIR}' folder, then cd back to '${APPDIR}' and enter `./AppRun`. When satisfied, repeat the "Create image" step above.

To use the --writableRoot option with the 0ad appimage, it must be extracted by providing the --appimage-extract option to the 0ad appimage. Then cd into squashfs-root, and type in `./AppRun --writableRoot`.

== *nix build ==
Linux build process shouldn't change (much).

See BuildInstructions#Linux

----
= Release, packaging and deployment =
The current release process is documented in ReleaseProcess.
Philip does that, except not the PPA packages or the Gentoo ebuilds. Ricotz creates the Ubuntu packages in an unknown way and other(unknown) people do it for Gentoo and the other distros.

Some information about OS-specific aspects of the packages... 


== Where to place which files ==
NOTE: This is just a draft.

Discussion about the Windows specific paths can be found in [http://www.wildfiregames.com/forum/index.php?showtopic=14759 this thread].

||'''Variable'''||'''Default'''||
||XDG_CACHE_HOME||~/.cache||
||XDG_CONFIG_HOME||~/.config||
||XDG_DATA_HOME||~/.local/share||

For release packaging, `update-workspaces.sh` can be used to set compile-time, absolute paths used by the game (to override defaults).
It should now offer a parameter for each path in the table below and not just for bindir, libdir and datadir (to be decided for each of those three if they should be replaced or kept):
 * `--bindir=/path/to/bin` sets the location of the pyrogenesis executable (eg. /usr/bin or /opt/0ad/system). Defines the `INSTALLED_BINDIR` preprocessor directive.
 * `--libdir=/path/to/lib` sets the location of the bundled libraries (eg. /usr/lib). Defines the `INSTALLED_LIBDIR` preprocessor directive.
 * `--datadir=/path/to/data` sets the location of the game's data (e.g. /usr/share/0ad/data, /usr/local/share/0ad/data, or /opt/0ad/data). Defines the `INSTALLED_DATADIR` preprocessor directive.
 * ... See the list below

The default Atlas config dir (or file) is specified in source/tools/atlas/AtlasUI/Misc/DLLInterface.cpp and does not adhere to the values in Paths.cpp, but uses the basedir-spec (See #868)

||'''Name'''||'''Readonly'''||'''Description'''||'''Path Windows'''||'''Path Linux'''||'''Path Mac'''||'''Determined by/Set where?'''||
||Game data root||Y||Data-Directory for static game data||relative to EXE||`INSTALLED_DATADIR` or relative to binary||`[BundlePath]/Contents/Resources/data` or relative to binary||`m_rdata` in Paths.cpp||
||User data root||N||Root-Directory for data the user creates in-game (savegames, screenshots etc...)||`My Documents\My Games\0ad`||`$XDG_DATA_HOME/0ad`||`~/Library/Application\ Support/0ad`||`m_Data` in Paths.cpp||
||Savegame directory||N||Directory where savegames are saved to and loaded from||`My Documents\My Games\0ad\saves`||`$XDG_DATA_HOME/0ad/saves`||`~/Library/Application\ Support/0ad/saves`||As `[User data root]\saves` in GameSetup.cpp InitVFS||
||screenshot directory||N||Directory where screenshots are saved by the game||`My Documents\My Games\0ad\screenshots`||`$XDG_DATA_HOME/0ad/screenshots`||`~/Library/Application\ Support/0ad/screenshots`||As `[User data root]\screenshots` in GameSetup.cpp InitVFS||
||Default config||Y||Default config-file which is readonly||relative to EXE||`INSTALLED_DATADIR/config` or relative to binary||`[BundlePath]/Contents/Resources/data/config` or relative to binary||As `[Game data root]\config` in GameSetup.cpp InitVFS||
||Userconfig||N||Contains user specific configuration settings||`%appdata%\0ad\config`||`$XDG_CONFIG_HOME/0ad/config`||`~/Library/Application\ Support/0ad/config`||`m_config` in Paths.cpp||
||Logs||N||Contains logs created by the game||`%appdata%\0ad\logs`||`$XDG_CONFIG_HOME/0ad/logs`||`~/Library/Application\ Support/0ad/logs`||`m_logs` in Paths.cpp||
||Cache||N||Cache directory||`%localappdata%\0ad\cache`||`$XDG_CACHE_HOME/0ad`||`~/Library/Caches/0ad`||`m_cache` in Paths.cpp||
||User mod||N||User mod directory, where e.g. Atlas should save maps||`My Documents\My Games\0ad\mods\user`||`$XDG_DATA_HOME/0ad/mods/user`||`~/Library/Application\ Support/0ad/mods/user`||As `[User data root]\mods\user` in GameSetup.cpp InitVFS||
||Mods||N (Y if running without -noUserMod)||Other mods added by the user||`My Documents\My Games\0ad\mods`||`$XDG_DATA_HOME/0ad/mods`||`~/Library/Application\ Support/0ad/mods`||As `[User data root]\mods` in GameSetup.cpp InitVFS||
----

= Open tasks =

== 1. Git migration ==
Assigned: unassigned / collaborate with Philip

To improve development workflow, we will migrate from SVN to Git.

How this affects the build system:
 * Autobuilder will be changed to build Git branches, allowing artists and others to test feature branches (see #1819)
 * Windows binaries will be moved out of the repository, starting with binaries/system, and possibly later including libraries/win32, etc. An update utility will be written to fetch the latest Windows binaries for the current branch (see #1814).

== 2. Change how the game handles default search paths ==
Assigned: unassigned

Each OS should have its default behaviour that should make sense for that OS (see the table above).
For Windows and Linux that's already quite good, but OSX requires improvements.

It should be distinguished between three different situations. 

 1. The current situation in development where the pyrogenesis binary is in 0ad/binaries/system, data is in 0ad/binaries/data etc... 
 1. The situation when everything is in a bundle and the directories are organized as described in "Where to place which files?" in this article.
 1. Special situations where the users wants to use customized directories. It should be sufficient to solve this with preprocessor directives because if anyone want's to change it, it's either an advanced user or package maker anyway and generally there should be no need to change it. Each defined location in the table above should be customizable with one dedicated preprocessor directive. 

Documentation to be updated:
 * [wiki:Finding_Your_Way_Around]
 * ... what else?

== 3. Check CMake as an alternative in the future or work on improved XCode support for Premake ==
Assigned: unassigned
Past work on an integration of CMake: #1104

----