#6676 closed defect (fixed)

python 3.11 build fix

Reported by: fabio Owned by: Itms
Priority: Release Blocker Milestone: Alpha 27
Component: Core engine Keywords:
Cc: Patch:

Description

Change History (9)

comment:1 by Ralph Sennhauser, 16 months ago

This fix is still required for SM91

Build just failed using python3.11 with

ValueError: invalid mode: 'rU'
ERROR: SpiderMonkey build failed

comment:3 by Vincent Cheng, 16 months ago

Hello, Debian packager here! To provide more context on how we got 0ad to build on Debian with python 3.11, you basically need 2 additional patches on top of spidermonkey:

  • a patch to fix the deprecated "U" file mode in python 3.11 [1]
  • a patch to upgrade virtualenv, cherrypicked upstream, which

unfortunately includes binary blobs and I couldn't find a way around that (see [2], and note that this is byte-for-byte the same as [3])

You can refer to the commit mentioned in the description to see how we applied both patches to patch.sh. I don't think you can work around having to upgrade virtualenv via that binary patch unless you wanted to repackage the spidermonkey source tarball (and hence also the 0ad source tarball - if you do this as upstream please don't re-release the upstream tarball with the same version number). Hope this helps!

[1] https://salsa.debian.org/games-team/0ad/-/blob/7048ef33282782d9af46335c9d928dfa9d9f379d/debian/patches/fix_python_3.11_ftbfs.patch

[2] https://salsa.debian.org/games-team/0ad/-/blob/7048ef33282782d9af46335c9d928dfa9d9f379d/debian/patches/mozjs_virtualenv.patch

[3] https://github.com/mozilla/gecko-dev/commit/74641307d32a59806b75cd2b8c7161aca50d5cb7

comment:4 by Ralph Sennhauser, 16 months ago

 [3] ​https://github.com/mozilla/gecko-dev/commit/74641307d32a59806b75cd2b8c7161aca50d5cb7

Updates to virtualenv-20.0.31 while sm91 already is at virtualenv-20.2.2

So might be a different issue Debian users are facing than with sm78

comment:5 by Stan, 16 months ago

In 27442:

Fix a bug with Python >= 3.11 where -rU was removed as an open flag.
Patch by: @sera
Refs #6676 Phab:rP27409
Differential Revision: https://code.wildfiregames.com/D4888

comment:6 by Stan, 16 months ago

From the Fedora maintainer:

We already carry the same "rU => "r" patch in Fedora, https://src.fedoraproject.org/rpms/0ad/blob/rawhide/f/0001-Python-Build-Use-r-instead-of-rU-file-read-modes.patch so getting it upstream into 0ad helps us as well :)

As for the virtualenv patch, that's a backport from upstream firefox 78 branch that fixes the build with newer setuptools that we have in Fedora. I don't know if forcing a virtualenv version would work like you suggest; I don't really understand the python build system enough for that.

What would really help downstreams would be to try to keep the bundled mozjs updated to the latest point release, which would be 78.15.0 right now. At least we could drop the crazy virtualenv patch then as it's all upstream in firefox 78 branch. But I also understand why you don't want to do it because checking in a 100MB firefox point release tarball to 0ad svn every 2 months (or whatever the firefox ESR release schedule is) would blow up the repo size quite fast :)

EDIT: Bugzilla link: https://bugzilla.mozilla.org/show_bug.cgi?id=1654457

EDIT2: Virtual env update https://phabricator.services.mozilla.com/rMOZILLACENTRAL3fbd3ae18e3fb25f11f0add79947ba3417dcdc3a

Last edited 16 months ago by Stan (previous) (diff)

comment:7 by Stan, 16 months ago

In 27445:

Fix typo noticed by @Itms
Refs #6676 Phab:rP27409
Differential Revision: https://code.wildfiregames.com/D4888

comment:8 by Ralph Sennhauser, 16 months ago

91.13 is the final point release, so using --with-system-mozjs is actually reasonably possible without having to worry it breaking 0ad the next day due to an update to mozjs.

Built with and ran with --with-system-mozjs on Gentoo successfully.

comment:9 by Itms, 16 months ago

Owner: set to Itms
Resolution: fixed
Status: newclosed

In 27460:

Fix and cleanup the SM91 build, fixes #6676.

Add a new patch written by s0600204, replacing the use of distutils with sysconfig in the setup of the bundled virtualenv in SpiderMonkey.
Add a patch which is a portion of https://phabricator.services.mozilla.com/D130410, fixing upstream's bug 1739486 on Fedora.

This fixes build issues on recent Debian and Fedora. This is consistent with upstream's upcoming changes in ESR102 following the deprecation of distutils.

This cleans up some old SM patches; renames the r27442 unicode patch; moves a Python-related environment variable inside build.sh; fixes indentation and EOL properties.

Differential Revision: https://code.wildfiregames.com/D4893

Note: See TracTickets for help on using tickets.