GnuTLS build failed on macOS Mojave

[Krinkle]

Hi there,

I'm just getting started with ​https://trac.wildfiregames.com/wiki/BuildInstructions#OSX. I've cloned the repo using svn (already had Arcanist installed for another project), and installed cmake from Homebrew.

During the build-osx-libs.sh step, I get the following error:

Building GnuTLS...
~/Development/0ad/libraries/osx/gnutls/gnutls-3.5.19 ~/Development/0ad/libraries/osx/gnutls ~/Development/0ad/libraries/osx
checking build system type... x86_64-apple-darwin18.5.0
checking host system type... x86_64-apple-darwin18.5.0
checking for a BSD-compatible install... /usr/local/opt/coreutils/libexec/gnubin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/local/opt/coreutils/libexec/gnubin/mkdir -p
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
***
*** Checking for compilation programs...
checking for pkg-config... no
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ISO C89... none needed
checking whether clang understands -c and -o together... yes
checking for style of include used by make... GNU
checking dependency style of clang... gcc3
checking how to run the C preprocessor... clang -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking whether _XOPEN_SOURCE should be defined... no
checking for Minix Amsterdam compiler... no
checking for ar... ar
checking the archiver (ar) interface... ar
checking for ar... (cached) ar
checking for ranlib... ranlib
checking for _LARGEFILE_SOURCE value needed for large files... no
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking dependency style of clang... gcc3
checking the archiver (ar) interface... (cached) ar
checking whether we are using the GNU C++ compiler... yes
checking whether clang++ accepts -g... yes
checking dependency style of clang++... gcc3
checking for bison... bison -y
checking for a sed that does not truncate output... /usr/bin/sed
checking for clang option to accept ISO C99... none needed
checking whether to build with code coverage support... no
checking whether to enable maintainer-specific portions of Makefiles... yes
checking for autogen... :
configure: WARNING:
***
*** autogen not found. Will not link against libopts.
*** 
checking for inline... inline
checking for ANSI C header files... (cached) yes
checking cpuid.h usability... yes
checking cpuid.h presence... yes
checking for cpuid.h... yes
checking for struct iovec.iov_basea... no
checking netinet/tcp.h usability... yes
checking netinet/tcp.h presence... yes
checking for netinet/tcp.h... yes
checking stdatomic.h usability... yes
checking stdatomic.h presence... yes
checking for stdatomic.h... yes
checking for getrandom... no
checking for getentropy... yes
checking for NETTLE... configure: error: 
  ***
  *** Libnettle 3.1 was not found.
ERROR: GnuTLS build failed

[wraitii]

Hello :) I think that guide might be somewhat out of date.

The error message is that lib nettle wasn't found. Could you try brew install nettle and see if it still fails?

[Krinkle]

Ah, okay, so nettle isn't one of libs it is meant to install locally with the script. No problem.

$ brew install nettle

Warning: nettle 3.4.1 is already installed and up-to-date

Looks like I had it already. I don't recall doing that though, so it was probably a dependency for something else. Anyway, just in case:

$ brew reinstall nettle
==> Reinstalling nettle 
==> Already downloaded: ==> Pouring nettle-3.4.1.mojave.bottle.tar.gz
🍺  /usr/local/Cellar/nettle/3.4.1: 85 files, 2MB

And re-running the 0ad lib-osx script {{ … * autogen not found. Will not link against libopts. checking for inline... inline checking for ANSI C header files... (cached) yes checking cpuid.h usability... yes checking cpuid.h presence... yes checking for cpuid.h... yes checking for struct iovec.iov_basea... no checking netinet/tcp.h usability... yes checking netinet/tcp.h presence... yes checking for netinet/tcp.h... yes checking stdatomic.h usability... yes checking stdatomic.h presence... yes checking for stdatomic.h... yes checking for getrandom... no checking for getentropy... yes checking for NETTLE... configure: error:

* Libnettle 3.1 was not found.

ERROR: GnuTLS build failed }}}

I guess it requires the older version? I'd rather not downgrade the system wide install, though.

[Stan]

The game's build system isn't compatible with brew yet. So I guess something goes wrong when downloading the dependencies with the script (which is what brew would usually be used to do)

Can you post the full build-osx-libs.sh output somewhere ?

[wraitii]

Er, we actually download Nettle in the 0 A.D. build scripts, but we download nettle 3.4 . So scratch what I've said above.

According to ​https://stackoverflow.com/questions/9508851/how-to-compile-gnutls this might actually be caused by how we compile nettle.

Indeed I would need the full build-osx-libs output here probably.

[Krinkle]

Yeah, I just reached the same conclusion. I don't think we need compatibility with Homebrew here. I like that we keep it all locally, and the script already has a section to install Nettle, right before the section for GnuTLS. But, similar to Homebrew, it too is installing version 3.4, not 3.1.

I've removed the nettle directory to let it re-run for output sharing. Pasted at ​https://code.wildfiregames.com/P151.

Best I can tell it is working fine, but indeed, the wrong version.

[Krinkle]

If feasible, I'd like the fix for this (if anything is needed) to be my first patch :)

[wraitii]

I can't reproduce when setting MIN_OSX_VERSIOn to 10.14 on my 10.14 mac. I can't actually compile GMP with the default setting there...

[elexis]

Duplicate at #5481

[kali0ad]

Not OP, but I have the same error. MIN_OSX_VERSION=10.14 ./build-osx-libs.sh -j3 --force-rebuild did not work for me. It generated about 6mb of logs, which I can't upload here, but here is the part that pertains to Nettle: ​https://pastebin.com/Fv7G7qpg. It also sent a bunch of output to the console which I pasted here: ​https://pastebin.com/ByF8wTep

[kali0ad]

I tried the patch here: ​https://code.wildfiregames.com/D2057

But then I got the following error:

Building GnuTLS...
Downloading gnutls-3.6.8.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   184    0   184    0     0    150      0 --:--:--  0:00:01 --:--:--   151
tar: Unrecognized archive format
tar: Error exit delayed from previous errors.

I think it was downloading a 404 or 500 response from the server instead of the archive.

So, I changed LIB_URL="https://www.gnupg.org/ftp/gcrypt/gnutls/v3.5/" to LIB_URL="https://www.gnupg.org/ftp/gcrypt/gnutls/v3.6/" in /libraries/osx/build-osx-libs.sh

I ran ./build-osx-libs.sh -j3 --force-rebuild but still got:

Building GnuTLS...
tar: Unrecognized archive format
tar: Error exit delayed from previous errors.

I then deleted /libraries/osx/gnutls and reran ./build-osx-libs.sh -j3 --force-rebuild and it worked!

[historic_bruno]

Thanks, that line didn't make it into my patch :/ (the download error page ends up being saved as the tar.gz, that's why it fails. This should be handled better...)

[historic_bruno]

In 22455:

Fixes GnuTLS build on macOS.
Fixes macOS linker warning "PIE disabled absolute - addressing not allowed".

Updates nettle to 3.5.1, GnuTLS to 3.6.8, gloox to 1.0.22.
Disables TCP fast open feature of GnuTLS (requires 10.11, no SDK build support).
Fixes GnuTLS detection of GMP by adding it to LIBS flag.
Disables getaddrinfo on gloox 1.0.22. Lobby connections failed during server hostname resolution.
Adds --with-pic to GMP build to force consistent PIC usage.
Adds -N flag to patch commands to avoid reapplying them.
Removes unneeded build flags.
Documents --enable-fat configure flag: GMP and nettle detect CPU-specific features, fat binaries let us build and run them on different CPUs (see D1772).

Fixes #5453, 5489. Refs #5481.
Tested by: kali0ad, trompetin17
Reviewed by: elexis, trompetin17

Differential Revision: ​​https://code.wildfiregames.com/D2057