[PATCH] clang error on FreeBSD

[Guido Falsi]

Hi,

I'm the maintaining the 0ad port in the FreeBSD ports collection.

I've been unable to make it compile after alpha 14. So I could not update the port further than that.

As you may know FreeBSD switched to clang as it's default compiler recently. Also the boost libraries are now being compiled using clang on all supported FreeBSD releases and being linked to libc++, not libstdc++ from gcc.

This means that to get a working 0ad I need to compile it with clang and link against libc++. I made a few patches to try to make it compile, but I'm not able to solve the error in the attached file. Anyone can help? Any patches welcome to make it compile on FreeBSD, I can use those as local patches while a permanent solution is found.

Full log of the failed build is available here (it's big...):

​http://www.madpilot.net/~mad/0ad/0ad-0.0.16.log.xz

Also, after release 10.0 FreeBSD includes an iconv implementation in it's libc, I had to patch the buid system to cater for this.

I also found an hardcoded gcc command in premake4.lua, I worked around that using substitutions.

The present state of the not working port is available here:

​http://www.madpilot.net/~mad/0ad/0ad.txz

It also contains all the additional patches I made, which I'd like to have included, after adaption obviously (some parts are strictly FreeBSD ports specific ones)

I hope someone will be willing to have a look at the patches and integrate them. I'm available for any explanations needed.

Thanks in advance for any help and/or feedback.

[Guido Falsi]

Trimmed clang error

[historic_bruno]

Seems like the recent work for OS X compatibility might help you, too (clang is now Apple's supported compiler and it defaults to libc++).

We are weeks away from A17 release, so I would focus on the SVN build, or even better fork our git repo: ​https://github.com/0ad/0ad. With git, it will be easier to see what you're doing and merge the parts we want, it will be better collaboration.

[Guido Falsi]

Replying to historic_bruno:

Seems like the recent work for OS X compatibility might help you, too (clang is now Apple's supported compiler and it defaults to libc++).

Thanks for the feedback.

I've been looking closely at OS X fixes and they have been handy, but I could not find any commit addressing the problem I encountered in EnvironmentHandlers.cpp. Anyone willing to help with that? My C++ knowledge is really basic, and I've been unable to find a fix for that error.

FreeBSD head(development version) uses clang 3.4.1, all other versions use clang 3.3.

We are weeks away from A17 release, so I would focus on the SVN build, or even better fork our git repo: ​https://github.com/0ad/0ad. With git, it will be easier to see what you're doing and merge the parts we want, it will be better collaboration.

I'm not a git user, I'll have to study a little before doing the fork.

I can make a fork and import my changes which I can port in a compatible way.

[historic_bruno]

Stop by our IRC channel (​#0ad-dev on QuakeNet and someone can probably walk you through setting up and using git :)

As for the build error, are you building with c++11 or using a library as such? I recall getting some errors like that in Atlas message passing on OS X, when using the -std=c++11 flag. It may be a bug that needs fixing, but dropping that flag fixed the problem for me. Philip` in IRC would be a good one to talk to about that, if you manage to catch him.

The other thing is you may be mixing libstc++ and libc++ or clang and gcc, do you have both available on your system?

[Guido Falsi]

Got on IRC, anyway I'm reading some github/git documentation, I should be up and running easily.

What I'm doing is maintaining a port in the FreeBSD ports collection, which is an organized Makefiles system to allow easy compilation of binary packages from sources. You can find documentation here if interested: ​http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports.html

To test my work I'm using a software called poudriere, which is the same software the FreeBSD project uses to build binary packages. It ensures the build happens in a clean environment.

Compilers situation in FreeBSD is...complicated, the migration to clang happened with version 10.0. Older versions bundled an old gcc release (4.2.1) and on those it coexists with clang which, if needed, is installed from the ports tree. On 10.0 and later by default only clang is installed in base, other compilers can be installed, if gcc is installed from ports it's installed as gcc48(for example). No "gcc" binary is created to avoid confusion. But for now I'd be happy to have a newer 0ad release work on FreeBSD 11(development version) or 10.x, I can deal with making it work on older releases once I succeed with that.

Also, versions of FreeBSD after 10.0 have no libstdc++ installed, unless it's installed by the user via ports. I just tried forcing gcc to build 0ad but, due to such mixing, I had many more problems than using clang, so I'll stick to clang.

I was needing -std=c++11 otherwise the build stops while building some libraries before even starting to compile the 0ad sources. But I can test removing the flag and check what happens.

I'm going to check if I can find some help on the IRC channel too.

Thanks.

[historic_bruno]

Replying to madpilot:

What I'm doing is maintaining a port in the FreeBSD ports collection, which is an organized Makefiles system to allow easy compilation of binary packages from sources. You can find documentation here if interested: ​http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports.html

leper and I worked on getting the game to build and run on *BSD a few years ago, so we're somewhat familiar with this :)

Also, versions of FreeBSD after 10.0 have no libstdc++ installed, unless it's installed by the user via ports. I just tried forcing gcc to build 0ad but, due to such mixing, I had many more problems than using clang, so I'll stick to clang.

That sounds almost as much fun as building on OS X Mavericks / recent Xcode, only Apple was "nice" enough to still include libstdc++ and a symbolic link to clang called "gcc" with some different build flags, as if that is equivalent...

I was needing -std=c++11 otherwise the build stops while building some libraries before even starting to compile the 0ad sources. But I can test removing the flag and check what happens.

To fix this single error, you can change line 149 of source\tools\atlas\GameInterface\Handlers\EnvironmentHandlers.cpp​ to:

        m_NewSettings = msg->settings._Unwrap();

but on OS X, there were still many errors. I don't have the C++/11 expertise to solve them. Philip may be one of the few on our team who does, and he wrote most of the Atlas code besides. He's not very active anymore, if you ask a concise question with the error message(s) included, there's a chance he will respond. He doesn't follow Trac or the forums or IRC logs anymore though. Yves is someone else you might ask, he is more active.

[Guido Falsi]

I have made two test runs with this patch applied, one with -std=c++11 and one without.

Anyone interested can grab this archive:

​http://www.madpilot.net/~mad/0ad/0ad-buildlogs.txz

It contains both logs, it decompresses to 2 files, one almost 20 MB, the other a little more than 10.

[leper]

Patch for FreeBSD 10.0

[leper]

Attached a patch that makes the game (svn version) build on FreeBSD 10.0 (with clang and libc++). Using wx3.0 and without -std=c++11.

There are some TODOs in there which I'll try to fix after setting up OpenBSD again, so I don't break the build there.

Dependencies obtained using pkgng (BuildInstructions update incoming).

Doesn't solve #1325 (I used the patch there to build though) and still has two test failures (#1142, #2780).

[Guido Falsi]

Replying to leper:

Attached a patch that makes the game (svn version) build on FreeBSD 10.0 (with clang and libc++). Using wx3.0 and without -std=c++11.

Thanks, I can confirm that I also suceeded on FreeBSD 11 (clang 3.4.1 and libc++) and wx 2.8, also without the -std=c++11 flag.

Clang 3.4.1 is slightly more picky and was not digesting line 577 in build/premake/premake4/src/host/lua-5.1.4/src/lauxlib.c (attaching patch file for this small thing).

Previous similar error is averted by the comment, I think.

I also made the test using the ports infrastructure, so once the distribution is available it should work easily, and I'll have to tweak only the installation procedure.

I'll test digesting the instructions in ​http://trac.wildfiregames.com/browser/ps/trunk/source/tools/dist/build.sh to create a distribution file to fully test it, installation included, I'll ask for help if needed.

There are some TODOs in there which I'll try to fix after setting up OpenBSD again, so I don't break the build there.

Dependencies obtained using pkgng (BuildInstructions update incoming).

Doesn't solve #1325 (I used the patch there to build though) and still has two test failures (#1142, #2780).

Using patch in #1325 too.

Thanks a lot for your work!

[Guido Falsi]

patch needed to avoid error with clang 3.4.1

[Guido Falsi]

Replying to madpilot:

Clang 3.4.1 is slightly more picky and was not digesting line 577 in build/premake/premake4/src/host/lua-5.1.4/src/lauxlib.c (attaching patch file for this small thing).

This was a false positive, it's just a warning. I was confused by another error just after that in the build log.

Sorry for the noise.

[Guido Falsi]

Please note that also the line of code in comment 8 is needed to make it compile on FreeBSD.

[Stan]

Can you make a full patch ?

[leper]

Replying to madpilot:

Please note that also the line of code in comment 8 is needed to make it compile on FreeBSD.

On FreeBSD 11? Can you build without c++11?

I'm currently testing on OpenBSD (also found a usage of bash where sh works just fine, which I'll include in #2810 (hit that issue while building on OpenBSD)) and will commit the updated patch.

[Guido Falsi]

Replying to leper:

Replying to madpilot:

Please note that also the line of code in comment 8 is needed to make it compile on FreeBSD.

On FreeBSD 11? Can you build without c++11?

Yes, I have successfully built it on 11, 10.0 and 9.3. It did not work on 8.4 due to a mistake I made, I'm testing again after fixing that. I'm quite confident it works on all FreeBSD versions. There are a few differences between the versions, I'm compiling using the ports tree, which takes care of most of them.

On all version it's being compiled using clang (from base system on 10 and 11, installed as a port on older versions), this is necessary since boost libraries are also being compiled using clang on all FreeBSD versions now.

One thing to note, on FreeBSD 10 and 11 it is not necessary to link with iconv (no -liconv on ld command line), it is instead necessary on older versions. Using the ports infrastructure I take care of that and patch extern_libs4.lua on the fly depending on the system version.

I don't know premake well. Is the OS version available in some variable I can test in extern_libs4.lua?

I'm currently testing on OpenBSD (also found a usage of bash where sh works just fine, which I'll include in #2810 (hit that issue while building on OpenBSD)) and will commit the updated patch.

Great! I saw bash being used there, but did not test if the script could do with plain sh.

I'll see if I can make a diff including all the changes discussed in this report. I just need to discover how to test the OS version in premake lua scripts.

[leper]

os.getversion().{release,major,minor} should be what you want (and os.getversion().description).

So you need c++11 because boost is compiled with it enabled? Or am I misunderstanding something?

[Guido Falsi]

Replying to leper:

os.getversion().{release,major,minor} should be what you want (and os.getversion().description).

I cooked up something but it's not pretty to look at. I'll try to discuss it in IRC before attaching the diff.

So you need c++11 because boost is compiled with it enabled? Or am I misunderstanding something?

No, I'm not using -std=c++11 and it's not needed.

What I was saying is that on all supported FreeBSD versions boost is compiled using clang by default and linked against libc++, so any software using boost also needs to be compiled using clang and most of all linked to libc++ on all versions of FreeBSD, even those where clang isn't the default compiler. So no need to worry about gcc getting in the equation.

[Guido Falsi]

cumulative diff with changes discussed

[Guido Falsi]

I added an attachment with the discussed changes, relative to the present subversion repository.

It also contains a check for FreeBSD 10 to know if linking against libiconv is necessary.

I also use another patch for spidermonkey in the FreeBSD port. I'm attaching it as a separate file, maybe it's too specific and should stay only in the port.

Also note that to compile on FreeBSD the patch in #1325 is also needed.

Thanks.

[Guido Falsi]

Patch to make spidermonkey compile with clang on FreeBSD

[leper]

New patch, should include everything. Please test.

madpilot: I wouldn't include the SpiderMonkey patch, but that should be included upstream (Alpha 18 is probably switching to SM31 so maybe that is already fixed, if it isn't please check with upstream and latest versions (those wouldn't work with 0 A.D. (might compile, but would be untested and possibly broken in strange ways) of SpiderMonkey)

vincent: Could you test this patch on Debian GNU/kFreeBSD? (Mostly to see if it doesn't fail to build because of iconv being defined in libc and it not finding libiconv)

[leper]

Consolidation of fixes for FreeBSD and OpenBSD; should also work on Debian GNU/kFreeBSD, but that hasn't been tested yet.

[Vincent Cheng]

svn trunk (r15792) with the above patch applied builds fine on a Debian kfreebsd porterbox. I haven't tried running the resulting binary, however.

[Guido Falsi]

Revised patch for BSD.

[Guido Falsi]

I found 2 minor problems. bsd_fixes_2.diff​ is a revised diff which adresses them.

First, lua said me:

Premake args: --with-system-miniupnpc --bindir=/usr/local/bin --datadir=/usr/local/share/0ad --libdir=/usr/ local/lib/0ad --collada --atlas ...work/0ad-0.0.17-alpha/build/premake/extern_libs4.lua:367: attempt to compare number with nil ERROR: Premake failed

Your check for FreeBSD 10 was: os.getversion().release >= 10 I changed that to: os.getversion().majorversion >= 10 (checked premake documentation, release is the bug fix/service pack number)

The I discovered that where you remove bash in FCollada Makefile, FreeBSD sh does not work. But FreeBSD comes with csh installed, which understands that syntax. So I added a check for FreeBSD there and used a variable to define the shell to be used.

Finally to compile on FreeBSD also the fix in comment 7 is needed, and also the freebsd10.diff​ patch added this line to libraries/source/fcollada/src/FCollada/FUtils/FUStringConversion.cpp:

template int FUStringConversion::ToInt32<char>(const char);

which is also needed along with the patch at #1325.

Anyway this patch is ok and greatly reduced the local patches needed to having a working FreeBSD port.

[leper]

In 15793:

Build fixes and improvements for BSDs. Fixes #2804.

[leper]

Replying to madpilot:

...work/0ad-0.0.17-alpha/build/premake/extern_libs4.lua:367: attempt to compare number with nil ERROR: Premake failed

Your check for FreeBSD 10 was: os.getversion().release >= 10 I changed that to: os.getversion().majorversion >= 10 (checked premake documentation, release is the bug fix/service pack number)

How did I get that wrong... It's majorversion, minorversion, revision (not release), I should know that having written the version code for *nix that's in premake.

The I discovered that where you remove bash in FCollada Makefile, FreeBSD sh does not work. But FreeBSD comes with csh installed, which understands that syntax. So I added a check for FreeBSD there and used a variable to define the shell to be used.

I'll leave bash in there in that case (you can still patch it if you want, but if it isn't working with sh everywhere I don't see the point of making it more complicated than it need be).

Finally to compile on FreeBSD also the fix in comment 7 is needed,

I still can't reproduce that, and from all comments related to that issue it is caused by using -std=c++11.

and also the freebsd10.diff​ patch added this line to libraries/source/fcollada/src/FCollada/FUtils/FUStringConversion.cpp:

How did that not end up in the patch, added back.

which is also needed along with the patch at #1325.

I'll not commit that one, but I think I'll add it back to the review queue, as the patch is working and I'm not fully in agreement with Redfox' reasoning there.

Anyway this patch is ok and greatly reduced the local patches needed to having a working FreeBSD port.

Happy packaging (and please report build issues here (on trac in another ticket) if they pop up).

[historic_bruno]

Replying to historic_bruno:

To fix this single error, you can change line 149 of source\tools\atlas\GameInterface\Handlers\EnvironmentHandlers.cpp​ to:

        m_NewSettings = msg->settings._Unwrap();

Fixed in r16157, with a hopefully better solution.