Opened 19 months ago

Last modified 2 months ago

#4362 new task

Update and cleanup OS X libraries

Reported by: fabio Owned by: Itms
Priority: Should Have Milestone: Alpha 24
Component: Build & Packages Keywords:
Cc: Patch:

Description (last modified by Itms)

Update and cleanup build-osx-libs.sh:

  • update libraries to latest releases (should do for alpha22, some have security issues);
  • boost system may not be needed anymore; remove it if confirmed;
  • verify how to use already built libpng for wxwidgets, rather than its builtin version (no need to provide two libpng, wx one is also older).

Change History (23)

comment:1 Changed 19 months ago by wraitii

I created a branch that relies on Homebrew to download all non-bundled libs (except for gloox, as we do not use ssl and brew does).

https://github.com/wraitii/0ad/tree/OSX_libs_brew

This seems to compile on my system (10.11.6), and has the advantages of:

  • cutting down compile time substantially, particularly for wxwidgets/boost
  • making sure we always use the latest versions

Now we voluntarily didn't use brew before. I'm not exactly sure why, but here are a few notes:

  • Brew is largely the best and most common OS X package manager now
  • It used to requires sudo at some point but no longer does
  • It does not introduce conflicts with OSX's libraries, as it can provide "unlinked" versions (say, for libpng)
  • it's got all the libraries we use.

I don't see why we should/could not rely on it now, at least.

NB: I removed libiconv because it seemed to be only used to compile the others, but I'm not actually sure about that.

Last edited 19 months ago by wraitii (previous) (diff)

comment:2 Changed 19 months ago by wraitii

Keywords: rfc patch added

comment:3 Changed 18 months ago by wraitii

As a sidenote: using brew libraries and hot linking seems to work but fails to compile into a binary that's self-sufficient, it'll probably take more changes. As a side-sidenote, wxwidgets 3.0.2 no longer compiles on macOs 10.12 (Sierra), you have to use 3.1.0

edit:hm actually not either, we'll have to port an upstream patch.

Last edited 18 months ago by wraitii (previous) (diff)

comment:4 Changed 12 months ago by Itms

Description: modified (diff)

Can we push that to A23 or is there an important library update that needs to be performed before packaging?

comment:5 Changed 12 months ago by fabio

Most (all?) libraries in build-osx-libs.sh should be updated for a22, some due to security issues.

Wraitii use of Homebrew can wait for a future release.

comment:6 Changed 12 months ago by fabio

Suggested for a22:

-ZLIB_VERSION="zlib-1.2.8"
-CURL_VERSION="curl-7.46.0"
-ICONV_VERSION="libiconv-1.14"
-XML2_VERSION="libxml2-2.9.3"
-SDL2_VERSION="SDL2-2.0.4"
-BOOST_VERSION="boost_1_60_0"
-WXWIDGETS_VERSION="wxWidgets-3.0.2"
+ZLIB_VERSION="zlib-1.2.11"
+CURL_VERSION="curl-7.54.0"
+ICONV_VERSION="libiconv-1.15"
+XML2_VERSION="libxml2-2.9.4"
+SDL2_VERSION="SDL2-2.0.5"
+BOOST_VERSION="boost_1_64_0"
+WXWIDGETS_VERSION="wxWidgets-3.0.3"
 # libpng was included as part of X11 but that's removed from Mountain Lion
 # (also the Snow Leopard version was ancient 1.2)
-PNG_VERSION="libpng-1.6.21"
+PNG_VERSION="libpng-1.6.29"
 OGG_VERSION="libogg-1.3.2"
 VORBIS_VERSION="libvorbis-1.3.5"
 # gloox is necessary for multiplayer lobby
-GLOOX_VERSION="gloox-1.0.14"
+GLOOX_VERSION="gloox-1.0.20"
 # NSPR is necessary for threadsafe Spidermonkey
-NSPR_VERSION="4.11"
+NSPR_VERSION="4.15"
 # OS X only includes part of ICU, and only the dylib
 # NOTE: remember to also update LIB_URL below when changing version
-ICU_VERSION="icu4c-56_1"
+ICU_VERSION="icu4c-59_1"
 ENET_VERSION="enet-1.3.13"
-MINIUPNPC_VERSION="miniupnpc-1.9.20151026"
+MINIUPNPC_VERSION="miniupnpc-1.9.20160209"

comment:7 Changed 12 months ago by Itms

Alright, I hope wraitii can look into that.

I'm going to look into performing a few updates on Windows as well, they are long due.

comment:8 Changed 12 months ago by elexis

Besides that zlib version not being available anymore (reported in #4639), nigel87 also uses OSX Sierra 10.12 which fails to build wxWidgets due to including quicktime which was dropped from that OS: http://trac.wxwidgets.org/ticket/17639 https://forums.wxwidgets.org/viewtopic.php?t=42856

He tried passing WXWIDGETS_VERSION="wxWidgets-3.0.3" and added --disable-qtkit for CONF_OPTS in L356, but that didn't help.

comment:9 Changed 12 months ago by elexis

In 19825:

Update OSX zlib version to the most recent and now only one hosted.

Fixes #4639
Refs #4362
Differential Revision: https://code.wildfiregames.com/D644
Tested By: Tobbi

comment:10 Changed 12 months ago by elexis

Keywords: rfc removed

comment:11 Changed 12 months ago by elexis

In 19848:

Update OSX libraries to the most recently released versions.

Remove wxWidgets patch from rP16155 that had been merged with the new release.

Differential Revision: https://code.wildfiregames.com/D679
Refs #4362
Based On Patch By: fabio
Discussed With: leper
Tested By: Tobbi, Itms

comment:12 Changed 12 months ago by elexis

Four exploits in the latest release of libxml2 mentioned in https://code.wildfiregames.com/D679?id=2703#inline-12860 should be either patched or it should be confirmed that we are not affected. Rest of the library versions should be checked for exploitable vulnerabilities too.

comment:13 Changed 12 months ago by elexis

Phab:D699 for the libxml2 snapshot.

comment:14 Changed 12 months ago by elexis

Milestone: Alpha 22Backlog

I set this to backlog because noone has the will to update libxml2 to the most recent dev snapshot before the alpha 22 release and because we will need a new ticket to keep track of updates for the next releases if this was closed as fixed.

comment:15 Changed 6 months ago by stan

for macosx, im encountering error: 'connectx' is only available on macOS 10.11 or newer while installing libraries on curl-7.54.0, fix is to update to curl-7.56.0, see discussion here: https://github.com/VCVRack/Rack/pull/200 quick fix is to update this line to : CURL_VERSION="curl-7.56.0" in libraries/osx/build-osx-libs.sh

Last edited 6 months ago by stan (previous) (diff)

comment:16 Changed 4 months ago by Itms

Keywords: patch removed
Milestone: BacklogAlpha 23
Owner: set to Itms

I am going to perform the curl change on macOS ASAP.

comment:17 Changed 3 months ago by Itms

In 21501:

Update libcurl for OSX, in order to fix the build. Use the opportunity to update libxml2 which had vulnerabilities.
Refs #4362

comment:18 Changed 3 months ago by Itms

Milestone: Alpha 23Alpha 24

comment:19 Changed 3 months ago by fabio

Given #4790 was recently closed, here is an update of latest libraries and current status (to be A23) for OS X:

Most are a bit behind and some are possibly security related. Patching is trivial, but it would likely need some testing.

Last edited 3 months ago by fabio (previous) (diff)

comment:20 Changed 3 months ago by fabio

  • curl 7.58 -> 7.59 fixes: CVE-2018-1000120, CVE-2018-1000121, CVE-2018-1000122
  • miniupnpc ​2.0.20180222 -> 2.0.20170509 fixes: "Fix buffer over run in minixml.c", "Fix uninitialized variable access in upnpreplyparse.c"
  • libvorbis 1.3.5 -> 1.3.6 fixes: CVE-2018-5146, CVE-2017-14632, CVE-2017-14633

Will update some low risk libraries here.

comment:21 Changed 3 months ago by fabio

In 21583:

Update some OS X libraries with security fixes and hopefully low regression risks. Refs #4362.

comment:22 Changed 3 months ago by fabio

Given #4790 was recently closed, here is an update of latest libraries and current status (to be A23) for OS X:

  • zlib: 1.2.11 - OK;
  • curl 7.59.0 - OK;
  • iconv: 1.15 - OK;
  • libxml2: 2.9.8 - OK;
  • libsdl2 2.0.6, 2.0.7. 2.0.8 - 2.0.5;
  • boost 1.66.0 - 1.64;
  • wxWidgets 3.0.4 - 3.0.3.1;
  • libpng 1.6.34 - OK;
  • libogg 1.3.3 - OK;
  • libvorbis 1.3.6 - OK;
  • gloox 1.0.20 - OK;
  • nspr 4.19 - 4.15;
  • icu4c 60.2 - 59.1;
  • enet 1.3.13 - OK;
  • miniupnpc 2.0.20180222 - OK.

Some are still a bit behind and possibly security related. Patching is trivial, but it would likely need some testing.

comment:23 Changed 2 months ago by Itms

In 21683:

Update libcurl to 7.59.0 on Windows and enable SSL support on Windows and macOS.
Refs #3004, #4362.

Note: See TracTickets for help on using tickets.