Access violation writing 0x00000000

[elexis]

Someone experienced a crash today playing a match with two others against 5 bots on r17251.

He was using windows 8.1, rejoined twice and crashed a long time after that.

Also notice Itms and wraitii also crashed few revisions ago.

Luckily we have a crashdump this time.

[elexis]

Files provided by Modefucka2.

[fatherbushido]

full commands.txt from player mrbushido

[wraitii]

If someone on Windows could post the .dmp file in a readable-by-all format, I would appreciate it.

[elexis]

Replaying that file visually or non-visually doesn't result in a crash. Despite an instant OOS error, the game could be replayed correctly (no idlers or the like).

[historic_bruno]

At a guess, he was out of memory and an allocation failed. It looks like a 1024x1024 (4MB) texture was being converted:

 	msvcr100.dll!_VEC_memcpy(void * dst=0x59b07040, void * src=0x00400000, int len=-1438227758)  + 0x51 bytes	C
 	nvtt.dll!527ba6d0() 	
 	[Frames below may be incorrect and/or missing, no symbols loaded for nvtt.dll]	
 	nvtt.dll!527d0abb() 	
>	pyrogenesis.exe!CTextureConverter::ConvertTexture(const std::shared_ptr<CTexture> & texture={...}, const Path & src={...}, const Path & dest={...}, const CTextureConverter::Settings & settings={...})  Line 451	C++
 	pyrogenesis.exe!CTextureManagerImpl::ConvertTexture(const std::shared_ptr<CTexture> & texture={...})  Line 317	C++
 	pyrogenesis.exe!CTextureManagerImpl::MakeProgress()  Line 413	C++
 	pyrogenesis.exe!RendererIncrementalLoad()  Line 257	C++
 	pyrogenesis.exe!Frame()  Line 328	C++
 	pyrogenesis.exe!RunGameOrAtlas(int argc=1, const char * * argv=0x00ce0d70)  Line 535 + 0x5 bytes	C++
 	pyrogenesis.exe!SDL_main(int argc=1, char * * argv=0x00ce0d70)  Line 579 + 0xb bytes	C++
 	pyrogenesis.exe!main(int argc=1, char * * argv=0x00ce0d70)  Line 140 + 0xd bytes	C
 	pyrogenesis.exe!wmain(int argc=1, wchar_t * * argv=0x00cd6150)  Line 380 + 0x9 bytes	C++
 	pyrogenesis.exe!__tmainCRTStartup()  Line 623 + 0x17 bytes	C
 	pyrogenesis.exe!CallStartupWithinTryBlock()  Line 397	C++
 	kernel32.dll!@BaseThreadInitThunk@12()  + 0x24 bytes	
 	ntdll.dll!__RtlUserThreadStart()  + 0x2f bytes	
 	ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes	

[Stan]

Hey historic_bruno, for the record how did you get that info. I tried to debug it with VS but I only got references to nvtt.dll and nothing to pyrogenesis. So I concluded it was texture related.

[historic_bruno]

Replying to stanislas69:

Hey historic_bruno, for the record how did you get that info. I tried to debug it with VS but I only got references to nvtt.dll and nothing to pyrogenesis. So I concluded it was texture related.

Do you have debug symbol paths set up in VS? You need to add the path to the game's binaries and they need to be the same version as the crash (r17251). Unfortunately we don't have debug symbols for NVTT.

[Stan]

Yup, but he wasn't able to link to pyrogenesis. This what I got ​http://imgur.com/D2ZnAmZ

refs #3609

[Josh]

Modern operating systems really shouldn't run out of memory, but It seems like adding proper allocation checking wouldn't be too difficult. In all of /source, we use malloc 135 times, realloc 52 times, and calloc 32 times. We just need to validate their return value is non-NULL. Unfortunately, finding other library functions which could allocate memory might be more tricky. On the upside, where new is used we can just catch the std::bad_alloc exception at the top of the hierarchy.

[historic_bruno]

On Windows, the 32-bit process has a 2GB limit, which the game can easily exceed due to memory leaks (and fragmentation!). Especially after playing multiple games without exiting. I haven't checked recently, but in the past it wasn't uncommon for a single game to reach 1.0-1.5GB.

[historic_bruno]

Replying to stanislas69:

Yup, but he wasn't able to link to pyrogenesis. This what I got ​http://imgur.com/D2ZnAmZ

refs #3609

Looks like newer VS handles the dump file differently than 2010.

[Stan]

Sounds annoying. Could you generate the pdb for newer release #3609 for nvtt ? I haven't been able to compile it :(

To come back on the issue, how comes someone converts textures in the middle of the game ?

[Josh]

Replying to stanislas69:

Sounds annoying. Could you generate the pdb for newer release #3609 for nvtt ? I haven't been able to compile it :(

To come back on the issue, how comes someone converts textures in the middle of the game ?

Texture conversion is done on-demand as models are loaded.

[Stan]

Replying to Josh:

Texture conversion is done on-demand as models are loaded.

I thought they were cached at some point

[ben]

In 17272:

Adds debug symbols for bundled win32 NVTT library, fixes #3609, refs #3603

[historic_bruno]

Replying to stanislas69:

Replying to Josh:

Texture conversion is done on-demand as models are loaded.

I thought they were cached at some point

They are cached after being converted, but how long would it take before you rendered every possible texture in-game? :) And if the texture was updated in SVN, that would invalidate the cached version.

The tricky thing about memory is the process may have actually had much more than 4MB available, but not in a contiguous block due to fragmentation, so the game was able to function normally until this new, relatively large texture was encountered.

[Stan]

Ah right, but that basically means we are likely to have (and there has been) a lot of those reports when testing S.V.N. on windows. Maybe switching to 64bits on compatible machines would alleviate some of those. Unless it's VRAM ?

We should have a map called texture_demo.xml to cache everything.

In this case though, I believe this is not really a release blocker, unless I'm missing something, or you want to get rid of bad malloc once and for all.

[historic_bruno]

I haven't see other reports, so I can't really compare. The next time someone gets an error like that (Access violation writing 0x00000000), it would be useful to inspect memory usage in task manager before exiting and to produce a ​VMMap report. Also report how long the game has been running, whether they played multiple games in one session, how many times they rejoined, etc.

[Stan]

I'm pushing to A20 even though its mostly a needinfo ticket.

[elexis]

Likely out-of-memory.