Upgrade Spidermonkey

[Yves]

Tickets related to the Spidermonkey upgrade:
#2126

We are currently using Spidermonkey 1.8.5. That's a very old version and newer versions have been improved in performance and stability (memory leaks fixed for example).

A few thoughts about the upgrade:

Version

Mozilla released version 17 which is from the ESR-series of Firefox (longer supported releases) and apparently they will continue releasing standalone versions of Spidermonkey for each ESR-release. Unfortunately some very interesting features like IonMonkey are only available in version 18. Another reason to upgrade to a newer version would be API changes that happen quite often. If we do a lot of work to upgrade now, it's better to already include newer API changes. My current experience from testing is that the API difference between version 17 and 18 probably isn't too big and we could get both versions running to compare them and decide which one to choose.

ESR17 release discussion: ​https://bugzilla.mozilla.org/show_bug.cgi?id=735599

JIT

We have already enabled JIT (Just In Time compiling) for release builds. Why do we even need to do it "Just in Time" when we (opposed to Firefox) know all scripts before we run them? Version 18 introduced a new JIT compiler called Ion Monkey. This compiler should further improve the optimization for compiled JS code.

There's an interesting blog-post about that topic (also other topics are interesting on that blog): ​https://blog.mozilla.org/javascript/2012/09/12/ionmonkey-in-firefox-18/

Garbage collection

Garbage collection and memory allocation is a major bottleneck for certain parts of JS code that update a lot of data very often. I made some measurements here: http://www.wildfiregames.com/forum/index.php?showtopic=16721&#entry262183

Garbage collection could be improved by improving the way we handle compartments, especially related to the AI. That's a bit vague but I'm still trying to figure out more about that. An interesting article: ​http://andreasgal.com/2010/10/13/compartments/ . That article is quite old, but I think the basic concepts explained there are still valid.

Garbage collection can now even be done in a separate thread or we could run it more often (each frame for example) to avoid lag-spikes when it happens. For the AI that problem would probably be solved anyway when we move it to a separate thread.

Since Firefox 16, incremental GC is supported. The question is if that even helps in our situation - probably not. ​https://blog.mozilla.org/javascript/2012/08/28/incremental-gc-in-firefox-16/

AI in a separate thread

AI calculations should be done in a separate thread. That way we don't have to worry if the AI needs a few hundred milliseconds to do its calculations and the framerate still stays smooth. I've noticed that this task is very much related to the Spidermonkey upgrade because a lot of threading-related aspects of Spidermonkey changed since v1.8.5.

[Yves]

I asked about the JIT topic and Ion-Monkey in the jsapi IRC channel. Logs ​here.

Summary: JIT compiled code can't be saved on the harddisk and used later because it contains memory addresses. However, a function should be compiled with Ion-Monkey when it has been run 1000 times, so everything that runs once per frame should be compiled after about 30 seconds with 30-35 fps, which should be fine for the moment.

Garbage collection occasionally throws away the JIT-ed code. To make sure this doesn't happen, calling "js::NotifyAnimationActivity" should be enough. Check ​Bug 753630 for more background information about that.

[Yves]

just for reference and discussion in the associate thread

[Yves]

just for reference and discussion in the associate thread

[ben]

In 13429:

Extends binary serializer to support some standard JS classes: Number, String, Boolean. Fixes #406.
Extends binary serializer to support typed arrays.
Extends binary serializer to support custom JS prototype objects in AIs, fixes #407.
Allows full serialization of AIs (not yet implemented). Refs #1089, #1886
Increases binary serializer script backref arena from 8 MB to 16 MB, refs #1842

[historic_bruno]

r13429 ref'd this ticket because I had to use some hacks to get around AIs sharing data between script contexts. Actually the solution could be much simpler if each AI's data was in a single script context (then ScriptInterface could be used to hold the serializable prototypes, and the implementation wouldn't be AI specific).

Also a heads up since probably these changes will need merging into your local repo, feel free to ask if anything causes trouble.