[PATCH] SDL2 Linux mouse cursor corruption

[fabio]

With SDL2 only I get this on main cursor and the "treasure" cursor. Other cursors (hourglass while map loading, in game cursors, ...) are fine. It happens always, in the menu as well in the game. I attach a photo, since screenshots (both with F2 and print screen) doesn't capture the cursor.

I noticed the cursor is ​trunk/binaries/data/mods/mod/art/textures/cursors/test.png?rev=15678, which is the same (md5sum) of ​trunk/binaries/data/mods/public/art/textures/cursors/arrow-default.png?rev=15678.

If I replace test.png with a different cursor, e.g.: ​trunk/binaries/data/mods/public/art/textures/cursors/action-unset-rally.png?rev=15678 I see the new cursor without corruption.

Maybe SDL2 notices the right part of the image is transparent and do some trick that make it shows as white?

[fabio]

SDL2 mouse cursor

[historic_bruno]

Which version of SDL2? I don't see this on Ubuntu 14.04 w/ SDL 2.0.2.

It may not be related, but looking at the documentation of ​SDL_CreateRGBSurfaceFrom, it claims the pixel data isn't copied and must last as long as the surface. Currently the code is creating a texture on the stack, passing it to surface, which gets passed to ​SDL_CreateColorCursor. I haven't noticed any problems with this, but it may be the cursor logic copies the pixel data and the surface is no longer needed (it could even be platform specific). Either way, it's a problem that should be fixed.

[fabio]

I am also using libsdl2 2.0.2 from Ubuntu 14.04, mine on a 32 bit system.

I have a mesa r300 gallium driver, but I get the same issue with gallium llvmpipe, which can be used with:

LIBGL_ALWAYS_SOFTWARE=1 ./0ad

[historic_bruno]

That's interesting, I tested on a VM and the graphics are Gallium 0.4 on llvmpipe:

Graphics Card  : VMware, Inc. Gallium 0.4 on llvmpipe (LLVM 3.4, 256 bits)
OpenGL Drivers : 2.1 Mesa 10.1.3; OpenGL 2.1 Mesa 10.1.3

Are you running the game fullscreen or windowed? Do you toggle fullscreen or change the window size before the bug occurs?

[fabio]

I reproduced the issue with my standard config (GLSL) and also when:

• disabling the 0ad cache (renaming the 0ad cache dir);
• using default config (renaming the 0ad config dir).

[fabio]

Just to make it clear that I always reproduced this issue, both when running in windowed mode, as well in full screen, which is the default config after cleaning the config dir.

[Josh]

I can reproduce this as well with open source drivers on AMD.

[historic_bruno]

Maybe it's fixed upstream. Do either of you want to try with an SDL2 Hg snapshot? ​https://www.libsdl.org/hg.php

You can override the default packaged SDL2 by setting configure's prefix flag to some local path (maybe in your home directory), building and installing (make && make install), exporting the SDL2_CONFIG variable with the path to the new installed sdl2-config, then update-workspaces.sh and rebuild the game.

[fabio]

I recompiled libsdl2 hg and replaced the system so with the compiled one (no 0ad rebuild): the problem is still here.

[fabio]

I noticed a similar issue also happens with the "treasure" cursor ​trunk/binaries/data/mods/public/art/textures/cursors/action-gather-treasure.png?rev=15678

More interestingly I noticed the original issue is also noticeable when converting the test.png cursor to a jpg version and then viewing it with an image viewer (eog). I also tried the tool pnginfo on both a good and a bad cursor, but the detected image parameters (resolution, depth, ...) are exactly the same.

[historic_bruno]

Possibly this: ​Color cursor transparency problem on Linux AMD? Unfortunately they didn't attach images, so I can't compare their results to yours. At least, it seems you're not the only one with SDL2 cursor issues on Linux/AMD drivers.

[elexis]

My cursor is scrambled in the exact same way too (Alpha18, Ubuntu 14.10, OpenGL 4, SDL2, AMD graphics card, fglrx driver from ubuntu repository)

[historic_bruno]

If you guys can report this on the SDL mailing-list/forum/bug tracker along with screenshots, it would be interesting to see if they have any advice and if it closely matches that existing report. If they really suspect it's a driver bug, the sooner it's reported, the sooner it can be fixed :)

[fabio]

I reported this issue on libsdl bugzilla as ​libsdl bug #2887.

[elexis]

Guess what - if I add

; Disable hardware cursors
nohwcursor = true

to my config file, the cursor problem is solved, for A17 and A18 with SDL2 (on A17 with SDL 1.2 it doesn't occur neither way).

[elexis]

One could automatically set nohwcursor = true for linux + AMD graphics cards on SDL2, until the real cause of the problem is solved.

[historic_bruno]

The problem with GL cursors is they are only rendered as fast as the game itself, so it gets very annoying during laggy times, but it's certainly a workaround if needed.

It may not be needed, there are updates to the SDL ticket and one idea involves modifying the cursor to avoid the bug (untested, see link).

[elexis]

thx historic_bruno, i was wondering why my cursor was lagging so badly suddenly on a17.... i prefer the scrambled fast cursor then...

[s0600204]

Opening the test.png cursor image in Gimp and using the eyedropper tool on the transparent areas, it appears that there is a band of white down the right-hand side of the image that isn't visible because those pixels happen to be also transparent.

If I may suggest something to try:

• Open up the test.png (why is it called that?) cursor image in Gimp (if you have it)
• From the File Menu, choose Export,
• Give it a different filename (but with the same extension), and hit Export
• When the Export Image as PNG dialog comes up, make sure the options Save Background color and Save values from transparent pixels are deselected/unchecked.
• Try this new cursor image in game.

Edit: I realise that this is sort of what's suggested/alluded to in an earlier link. Apologies. I should read links provided before speaking.

[elexis]

Works perfectly!

I had encountered this issue of transparent colored pixels being displayed in other software too...

[Stan]

Maybe submit it here :)

[historic_bruno]

Let's commit the "fixed" cursor(s) for A18. Is it only the default cursor that has colored transparent pixels? (anyone know a good way to check?)

[Stan]

If you give me the list of corrupted cursors i could fix them.

To see you need an image editor :)

[historic_bruno]

Opening 30+ PNGs in Gimp's awkward UI wasn't what I had in mind when I asked for a good way :P

[Stan]

I could do it for you :)

Also you could read byte per byte data in those file and check %16 for each colored alpha pixels x)

[fabio]

As I said in comment:12 there is at least another cursors with this issue. Also some cursors are duplicated between mod and public, they have the same file in different directory.

[elexis]

Found a way!

The tool "identify" of the image magick library can show a histogram.

use "identify -verbose filename.png" to output the histogram. It consists of lines like those:

         1: (  0,  0,  0, 20) #00000014 srgba(0,0,0,0.0784314)

If there is one line that contains a srga tuple that ends with 0, then there is a colored transparent pixel in that file, i.e. the file should be fixed.

[elexis]

This bash script identifies png files that need to be fixed.

[elexis]

My script tells me the following files need to be fixed.

action-gather-ruins.png
action-gather-treasure.png
action-heal-disabled.png
action-heal.png
action-herd.png
action-remove-guard.png
action-setup-trade-route.png
arrow-default.png
cursor-flare.png
cursor-no.png

I think image magick should be able to fix this one by command line too...

[elexis]

This script uses image magick to remove transparent colored pixels automatically.

[elexis]

Notice my fix script must be run in every directory.

Additional information about "dirty transparency" and removing the "garbage alpha channel" with image magick:

​http://www.imagemagick.org/Usage/masking/ paragraph "Alpha Background" ​http://www.imagemagick.org/discourse-server/viewtopic.php?t=13746

[elexis]

This script updates all cursors using image magick and creates a 7zip archive containing the changed files. Execute in trunk directory.

[elexis]

fixed cursors

[Stan]

If you run it in trunk it will edit standard texture files right ? And removing color on those is not what you want X)

[elexis]

As defined in line 12+13, it changes files only in:

subdir1=binaries/data/mods/mod/art/textures/cursors/;
subdir2=binaries/data/mods/public/art/textures/cursors/;

You can set arbitrary directories. Icons also seem to be affected, but I didn't look into it.

If actual 3D model textures are changed, there is a crash. Thats why I dont do that.

The cursors_fixed.7z above was produced by the script and contains all cursors that need to be fixed. Old files are simply renamed (you can of course revert with 'svn revert -R .' in the trunk directory easily.

I verified that the trasure icon is broken (as mentioned by other people), unless I apply that patch.

[Stan]

Oh yeah missed that :)

Great hopes it get commited soon :)

[elexis]

This file contains all png files with dirty transparency, i.e. other images that might be broken.

[Philip Taylor]

Maybe the problem is that we load the PNGs as unpremultiplied alpha (so (255,255,255,0) is fully transparent), but SDL or X11 or the graphics drivers assume the cursors are premultiplied alpha (so they will add (255,255,255) to the background, instead of (255,255,255)*0).

You could possibly test that by making a cursor that is white with 50% alpha. If it gets rendered as solid white, it's probably that problem. Also try making one that's 50% grey with 50% alpha; if it gets rendered as semi-transparent white (instead of semi-transparent grey) then it's probably that problem.

In that case, the fix would be for the code to convert our unpremultiplied images to premultiplied (either our code or SDL2, depending on how the API is defined).

In any case, there's nothing inherently wrong with the PNG files themselves - the problem is just with how they're being interpreted.

[fabio]

In 16386:

Fixes cursor with dirty transparency.

I used elexis script and also optimized pngs with optipng -o 5 and advdef -z -4

Thanks elexis for the imagemagick script!

Fixes #2823.

[elexis]

Side note: I tried to write a script that displays only those affected pixels. Unfortunately image magick doesn't seem to have a way to do that. I tried

convert  "${filename}" -fx 'a==0 && (r>0 || b>0 || g>0) ? Hotpink : u' -alpha off "${filename}_garbage.png";

but apparently all fully-transparent pixels are interpreted to have no color.

[fabio]

I didn't notice before, but after the fix the unused test-old.png file looks darker, while others looks OK. Do you have an idea about it?

[Stan]

That icon was probably having some white alpha colored pixels, the fix would have made them not transparent anymore.

[historic_bruno]

Replying to elexis:

Found a way!

The tool "identify" of the image magick library can show a histogram.

use "identify -verbose filename.png" to output the histogram. It consists of lines like those:

         1: (  0,  0,  0, 20) #00000014 srgba(0,0,0,0.0784314)

If there is one line that contains a srga tuple that ends with 0, then there is a colored transparent pixel in that file, i.e. the file should be fixed.

Awesome, thanks a lot for working on this and getting it done in time for A18 :)

[Philip Taylor]

Could someone with the affected drivers try the attached alpha-test.png as a cursor, and look at it on a very dark background (e.g. open the Manual window) and a very light background (e.g. the smoke)? and then take screenshots of it, but be careful to verify that the screenshot looks the same as you really see on the screen.

That should hopefully indicate whether it's a premultiplied alpha problem, or something different.

I have a mesa r300 gallium driver, but I get the same issue with gallium llvmpipe

I think the OpenGL implementation doesn't matter for hardware cursors - it's the display driver used by the X server. (Cursors are usually supported with special hardware overlay surfaces that the display will composite on the fly as it's copying the framebuffer out to the HDMI port, which is why they're smooth even if you're updating your framebuffer very slowly.)

[elexis]

The mouse cursor is not captured, or not captured correctly by any tool I tested (0ad screenshot, gnome-screenshot, gimp, simplescreenrecorder, vlc).

Gimp & simplescreenrecorder show a strangely colored version of the cursor.

I can still describe the cursor: On white background (smoke) the cursor is filled diagonally with a gradient from the bottom left (black) to the top right (white).

On 'dark background' (manual) the cursor is filled horizontally with a gradient from the left (black) to the right (white).

If I hover the cursor over white text on dark background, i can see on the left (dark) side of the cursor, that the transparency decreases from the top (very transparent) to the bottom (opaque).

[elexis]

My cursor looks exactly like those two on the bottom.

[Philip Taylor]

The top pair were rendered in GL with glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) and the bottom pair with glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA).

If the hardware cursor looks like the bottom pair, then the cursor is being blended as if it were premultiplied alpha, while the image data is actually non-premultiplied alpha. That sounds like a bug in probably the display driver (or maybe SDL).

The changes in r16386 will have fixed the most obvious artifact of that bug, but the cursors will still be wrong in any pixels with alpha!=255. But I don't think there's anything we can do about it in our code - I believe we're doing the right thing already, since it works correctly on most drivers.

This problem will only affect cursors, it's completely independent of any other textures the game uses (which all use OpenGL blending, which works correctly), so there's no need to change any other textures.

[fabio]

elexis, do you have idea why test-old.png was not be identified by you in comment:30, but when I run the script myself was identified and then converted to a darker (when viewed in a browser) version?

[fabio]

In 16387:

binaries/data/mods/mod/art/textures/cursors/test.png and binaries/data/mods/public/art/textures/cursors/arrow-default.png were previously a copy of the same file (same md5), during conversion in r16386 they were indipendently converted and had different md5 (just the date inside the png was different). Use again the same file just to make obvious they are identical.

Refs: #2823.

[elexis]

Replying to fabio:

elexis, do you have idea why test-old.png was not be identified by you in comment:30, but when I run the script myself was identified and then converted to a darker (when viewed in a browser) version?

In line 26 i skip all files that end on "_old.png", since i rename the old files to that. Feel free to change it. Also on comment 30 i checked only one directory.

    if [[ "$filename" == *_old.png ]]; then

I didnt check the brightness specifically, but I also didn't notice any obvious differences when comparing. Also i didn't do the optimizations u talked about in commend 37:

optipng -o 5 and advdef -z -4

[elexis]

The better patch would be to remove the redundancy in the first place and don't use two files for one cursor.

Replying to fabio:

In 16387:

binaries/data/mods/mod/art/textures/cursors/test.png and binaries/data/mods/public/art/textures/cursors/arrow-default.png were previously a copy of the same file (same md5), during conversion in r16386 they were indipendently converted and had different md5 (just the date inside the png was different). Use again the same file just to make obvious they are identical.

Refs: #2823.

Edit: Sorry didnt see u already committed

Refs: #2823. }}}

[fabio]

Replying to elexis:

The better patch would be to remove the redundancy in the first place and don't use two files for one cursor.

No idea how this would be possible, I think the two mods should be independent.

[fabio]

Replying to elexis:

In line 26 i skip all files that end on "_old.png", since i rename the old files to that. Feel free to change it. Also on comment 30 i checked only one directory.

    if [[ "$filename" == *_old.png ]]; then

That file ends with -old.png, not _old.png, also it's in the same dir. Using your find... script it also get reported as needed to be fixed

I didnt check the brightness specifically, but I also didn't notice any obvious differences when comparing. Also i didn't do the optimizations u talked about in commend 37:

optipng -o 5 and advdef -z -4

I just noticed that running

convert  "test-old.png" -fx 'a==0 ? 0 : u' "test-old_.png";

make it darker. Also here is a snip of the diff of identify before and after conversion:

   Units: PixelsPerCentimeter
-  Type: TrueColorAlpha
+  Type: GrayscaleAlpha
+  Base type: GrayscaleAlpha
   Endianess: Undefined
-  Colorspace: sRGB
+  Colorspace: Gray
   Depth: 8-bit

Probably the imagemagick trick doesn't work well for that file. I'll revert the change on that file.

[fabio]

In 16400:

Revert r16386 on unused test-old.png (and optimize with optipng and advdef anyway), apparently imagemagick trick mess it up. It could be eventually be removed if not needed.

Refs #2823.

[fabio]

opting also reduces it to gray+alpha, but the image output it's the same as before.

[historic_bruno]

Unfortunately r16386 has broken some cursors, notably action-heal by converting it to indexed color (#3082). test-old would also be broken since it's grayscale, more likely we don't even use it.

[ben]

In 16407:

Reverts action-heal cursor from r16386, it was modified to indexed color not currently supported by the engine, fixes #3082, refs #1640. Reapplies the transparent color removal, refs #2823.
Deletes unused test-old cursor

[ben]

In 16439:

Improves handling of PNG textures based on GdkPixbuf's setup_png_transformations. The PNG decoder should now support grayscale, RGB or indexed color; bit depth up to 16; and interlacing. Fixes #1640, refs #2823, #3082

[ben]

In 16440:

Restores action-heal cursor from r16386 now that the engine supports indexed color PNGs, refs #2823

[fabio]

In 17222:

Fix action-capture icon transparency.

Refs: #2823

[fabio]

In 17230:

Fix action-capture icon transparency and optimize with optipng -o 5 and advdef -z -4

Refs: #2823