SDL2 Support

[historic_bruno]

This ticket is for working on ​SDL2 support (first stable release on 2013-08-13). Doc for 1.2 -> 2.0 migration ​here.

Known problems:

• Text input has changed in SDL2 to better support CJK, see ​TextInput tutorial and this ​forum topic. Basically Unicode characters may take multiple key presses to generate and we're now expected to use the text input API. In particular, SDL_Keysym.unicode is used by all text input in the game, but that was deprecated and has now been removed in the latest RC.
• Using SDL on Windows conflicts with our hardware cursor implementation. This can be fixed by using SDL2's color cursors on all platforms, provided it works, #935 has a patch for that.
• WMI shutdown crashes, moving it before SDL shutdown fixes that.
• Atlas is broken on Linux, OpenGL functions fail in the engine thread (note the GL context is created by wxWidgets in the GUI thread)

[ben]

In 13572:

Fixes a build error with SDL2, refs #2041

[historic_bruno]

Replying to historic_bruno:

• Text input has changed in SDL2 to better support CJK, see ​TextInput tutorial and this ​forum topic. Basically Unicode characters may take multiple key presses to generate and we're now expected to use the text input API. In particular, SDL_Keysym.unicode is used by all text input in the game, but that was deprecated and has now been removed in the latest RC.

Looked into this a bit more, and it's not as significant a change as I was expecting. Text input events will typically contain only single characters in UTF-8 encoding, roughly the same as the old SDL_Keysym.unicode, so it needs a conversion from UTF-8 to wide chars, but also a loop in case of multiple characters per event (or we can assert that there is only one).

I added some text input event handling to CConsole and CInput, and got basic input working (with the nice bonus of supporting Alt codes in Windows, allowing entry of special Unicode characters not available on the keyboard). Now there is an issue with backquote (`) and tilde (~) opening the console when an input box has focus, this is probably due to the old keydown event being handled in addition to the new text input event. And it remains to be tested on Linux and OS X.

[historic_bruno]

Should test if SDL2 fixes #2146 as well as #1733.

[julian37]

Using SDL2 fixes #1593 in my tests.

@historic_bruno: where are your text input event handling additions?

I've made a rough first patch to build with SDL2 against r14495, it sounds like it overlaps what you've been doing but perhaps you find it useful.

[julian37]

Patch against r14495 to build with SDL2 (2.0.1)

[Niek]

It seems to me that OpenGL 2.1 is no longer supported with SDL2 / gives a lot of trouble? (See the featurelist in the migration doc)

Upgrading to OpenGL 3.0 could solve OpenGL-related issues.

[historic_bruno]

Replying to niektb:

It seems to me that OpenGL 2.1 is no longer supported with SDL2 / gives a lot of trouble? (See the featurelist in the migration doc)

Upgrading to OpenGL 3.0 could solve OpenGL-related issues.

Which OpenGL issues? Do you mean issues we have or issues in SDL? We don't use SDL for its graphics API, it only creates the GL context, as far as I know we can specify the version we want (and in fact our current SDL2 code path does so, with ​SDL_GL_SetAttribute). Any documentation that says we can't do what we're doing would be good to see :)

[historic_bruno]

Replying to julian37:

Using SDL2 fixes #1593 in my tests.

@historic_bruno: where are your text input event handling additions?

I've made a rough first patch to build with SDL2 against r14495, it sounds like it overlaps what you've been doing but perhaps you find it useful.

Rather than attaching a patch, I made a git branch: ​https://github.com/historicbruno/0ad/commits/sdl2-support

You can see there is indeed some overlap, some of the fixes are trivial to get it compiling. The biggest effort IMO will be getting text input and hotkeys (and places where both are needed) working the same as it does with SDL 1.2 - or at least to a point we can agree is correct and consistent.

[historic_bruno]

I would like this to get into A17, it fixes a number of bugs on Windows and makes the OS X experience nicer as well. I'm not sure if we want to require everyone to use SDL2 yet, so maybe it should be a premake/build option (like --with-sdl2) until we get more feedback. I can build and test the Windows libs, if others can test more thoroughly on Linux and OS X.

[historic_bruno]

Important things to note are behavior of cursors, hotkeys, text input, and multiple displays, in addition to basic compile and run testing.

[Philip Taylor]

I'm not sure if we want to require everyone to use SDL2 yet

I think we should continue to support Ubuntu 12.04 for the foreseeable future (since it's still supported by Ubuntu, and ~10% of our players since the start of this year were using it). The libsdl2-dev package wasn't added until 13.10, and bundling or backporting libraries is nasty, so maintaining SDL1 support in our code for the next couple of years is probably the less painful approach.

[fabio]

Replying to historic_bruno:

I can build and test the Windows libs, if others can test more thoroughly on Linux and OS X.

Is the github branch ready to be tested? That one is very old (2013).

[wraitii]

Moving to SDL2 fixes the OSX crash with Atlas entirely (see #2650).

[fabio]

BTW libsdl2 can be easily backported (from debian sid) to Ubuntu 12.04, I did it ​here. Also, there are already 5 backported packages for 12.04 in the PPA, see ​here, we could add another one.

[historic_bruno]

Replying to wraitii:

Moving to SDL2 fixes the OSX crash with Atlas entirely (see #2650).

Annoyingly, it broke Atlas on Linux, I just remembered that, but I didn't test with wxWidgets 3.0, and it's possible the latest SDL2 magically fixes that too.

[Stan]

Any update on this ? If we could fix atlas on all platforms, that would probably bring more interest, since we have triggers now

[historic_bruno]

​sdl2-support branch is now rebased and updated for latest SVN, with SDL 2.0.3 binaries for Windows testing. Pass the --sdl2 option to update-workspaces, this is untested on *nix so the paths may not be correct there.

I noticed a bug on Windows XP, I had the game open full screen and pressed Ctrl+Alt+Del a few times to open the task manager, then I couldn't get back into the game. There was also a crash (seems a GL init issue) when starting mods from the new mod selection screen - I don't know if that bug also affects SVN.

[yashi]

just tried sdl2-support branch and It Works for Me(tm). Still, a few small fixes need to be applied.

• build/premake/extern_libs4.lua still needs if _OPTIONS["sdl2"] block for the compile_settings in addition to the link_settings you fixed.

• source/ps/GameSetup/HWDetect.cpp::ReportGLLimits() needs to aware of SDL2, like:

  source/ps/GameSetup/HWDetect.cpp

  @@ -44 +44 @@
   
   #ifdef SDL_VIDEO_DRIVER_X11
   #include <GL/glx.h>
  -#include <SDL/SDL_syswm.h>
  +#include <SDL_syswm.h>
   
   // Define the GLX_MESA_query_renderer macros if built with
   // an old Mesa (<10.0) that doesn't provide them
  @@ -707 +707 @@
   
       SDL_SysWMinfo wminfo;
       SDL_VERSION(&wminfo.version);
  -    if (SDL_GetWMInfo(&wminfo) && wminfo.subsystem == SDL_SYSWM_X11)
  +#if SDL_VERSION_ATLEAST(2, 0, 0)
  +    const int ret = SDL_GetWindowWMInfo(g_VideoMode.GetWindow(), &wminfo);
  +#else
  +    const int ret = SDL_GetWMInfo(&wminfo);
  +#endif
  +    if (ret && wminfo.subsystem == SDL_SYSWM_X11)
       {
  +#if SDL_VERSION_ATLEAST(2, 0, 0)
  +        Display* dpy = wminfo.info.x11.display;
  +#else
           Display* dpy = wminfo.info.x11.gfxdisplay;
  +#endif
           int scrnum = DefaultScreen(dpy);
   
           const char* glxexts = glXQueryExtensionsString(dpy, scrnum);

Even with the above fixes, it still crash when switching from mod selector to the game. enabling the following config and by-passing mod selction let me play the game even with my multi-monitor settings! yes!

mod.enabledmods = "mod public"

[fabio]

Hey, I was trying it also and sent a pull request for the compile issue.

The crash issue if not sdl2 specific and is tracked at #2753.

[historic_bruno]

Replying to yashi:

Hi, thanks for testing on Linux! I notice you change the SDL include path, but I think it will currently work with SDL 1.2. Does your change break the SDL 1.2 build, is it necessary for SDL2 on Linux?

[fabio]

I can confirm that without the include change sdl2 fails to build. But with the change sdl1 fails, so it should also be ifdeffed only for sdl2.

[yashi]

Replying to fabio:

Hey, I was trying it also and sent a pull request for the compile issue.

Great. Sorry that I didn't notice your pull request on github.

The crash issue if not sdl2 specific and is tracked at #2753.

Thanks. Will comment on that ticket. It seems to me that the issue is related to SDL, though. window->magic does not match _this->window_magic after restart.

[historic_bruno]

Replying to fabio:

I can confirm that without the include change sdl2 fails to build. But with the change sdl1 fails, so it should also be ifdeffed only for sdl2.

It seems in other source files, we include SDL headers directly with no path, like #include "SDL_syswm.h" in x.cpp. I don't see why it wouldn't work there, can you try that? I'm stuck on Windows for the moment.

[yashi]

Replying to historic_bruno:

It seems in other source files, we include SDL headers directly with no path, like #include "SDL_syswm.h" in x.cpp. I don't see why it wouldn't work there, can you try that? I'm stuck on Windows for the moment.

It turns out that most Linux system has SDL's include directory under the system default /usr/include and gcc search there anyhow. Thus, regardless of the value returned from sdl-config --cflags, we find SDL/.... on ordinary systems.

I've installed SDL1 under some other directory, and it did fail without fix. So, it is a bug no one triggered yet.

BTW, is there any reason why we use "..." instead of <...>? Is it because it's not a system library under Windows?

[fabio]

Replying to historic_bruno:

Replying to fabio:

I can confirm that without the include change sdl2 fails to build. But with the change sdl1 fails, so it should also be ifdeffed only for sdl2.

It seems in other source files, we include SDL headers directly with no path, like #include "SDL_syswm.h" in x.cpp. I don't see why it wouldn't work there, can you try that? I'm stuck on Windows for the moment.

This is the error I am getting with yashi patch and SDL1 on debian testing:

../../../source/lib/res/graphics/cursor.cpp: In member function ‘Status SDLCursor::create(const PIVFS&, const VfsPath&, int, int)’:
../../../source/lib/res/graphics/cursor.cpp:104:63: error: ‘SDL_CreateColorCursor’ was not declared in this scope
   cursor = SDL_CreateColorCursor(surface, hotspotx_, hotspoty_);
                                                               ^

[fabio]

And I get the same error with "SDL_..." rather than <SDL_...>.

[historic_bruno]

Replying to fabio:

This is the error I am getting with yashi patch and SDL1 on debian testing:

../../../source/lib/res/graphics/cursor.cpp: In member function ‘Status SDLCursor::create(const PIVFS&, const VfsPath&, int, int)’:
../../../source/lib/res/graphics/cursor.cpp:104:63: error: ‘SDL_CreateColorCursor’ was not declared in this scope
   cursor = SDL_CreateColorCursor(surface, hotspotx_, hotspoty_);
                                                               ^

That isn't related to the includes - I thought I had tested the cursor code with SDL 1.2, but obviously not...

[historic_bruno]

Replying to yashi:

BTW, is there any reason why we use "..." instead of <...>? Is it because it's not a system library under Windows?

r10895 explains that briefly, in fact it addressed the very issue we're discussing :)

I've rebased sdl2-support to fix the SDL 1.2 build failures.

[fabio]

I can confirm current sdl2 branch compiles OK with both sdl1 and sdl2 on debian.

[historic_bruno]

Thanks!

[Niek]

Replying to historic_bruno:

Thanks!

But what about the error you posted?

[yashi]

The current sdl2-support branch seems to be working with both SDL1 and SDL2 on my system, however, I see two problems with SDL2.

First, with SDL2, mouse tilt does not work. This seems to be an API change on SDL. My mouse reports that button 6 and 7 when I tilt left and right, respectively.

Second, I see the map rotating clockwise when I zooming-out fast with my mouse wheel. Has anyone seen this?

[historic_bruno]

Replying to niektb:

But what about the error you posted?

The error restarting the engine after mod selection is fixed by yashi's patch (​cec49f5). Then the following are still issues:

• Text input interacts differently with hotkeys, this is most noticeable with the backquote !` key in a text input field. It is also a console.toggle hotkey, it will both add the backquote and open the console.
  • In SDL1, it ignores the hotkey as a special case and inputs a backquote.
  • SDL2 sends three events for this single key press (actually one is a custom event sent by the hotkey handler): key down, hotkey down, and text input. The events aren't connected in the API, and I don't know that I can count on their order.
  • I think similar problems would occur with other hotkeys that have visible input characters.
• In Windows, debugging the game in fullscreen is impossible, the window seems to maintain focus over VS. There may be other fullscreen focus-related issues.
• Atlas errors on Linux?

Replying to yashi:

First, with SDL2, mouse tilt does not work. This seems to be an API change on SDL. My mouse reports that button 6 and 7 when I tilt left and right, respectively.

What functionality is mouse tilt in the game?