Opened 7 years ago

Last modified 3 years ago

#1619 new defect

Big screenshot fails on win32 (Out of address space)

Reported by: historic_bruno Owned by:
Priority: Should Have Milestone: Backlog
Component: Core engine Keywords:
Cc: Patch:

Description (last modified by historic_bruno)

Able to reproduce this on Windows 7 64-bit, of course using a 32-bit build of the game. Big screenshots (Shift+F2) work for me at the start of games, but later on in the game it fails with this error:

Allocation failed: base=00000000 allocBase=00000000 allocProt=0 size=65536 state=65536 prot=1 type=0
AllocateLargeOrSmallPages of 371195904 failed
wvm.cpp(297): Out of address space (64-bit OS may help)


>	pyrogenesis.exe!PerformErrorReaction(ErrorReactionInternal er=255, unsigned int flags=0, volatile int * suppress=0x00000000)  Line 421	C++
 	pyrogenesis.exe!debug_DisplayError(const wchar_t * description=0x325be828, unsigned int flags=0, void * context=0x0074ef80, const wchar_t * lastFuncToSkip=0x00353acc, const wchar_t * pathname=0x00377aa0, int line=297, const char * func=0x00377d2c, volatile int * suppress=0x00000000)  Line 478 + 0xc bytes	C++
 	pyrogenesis.exe!vm::ReserveAddressSpace(unsigned int size=368902144, unsigned int commitSize=2097152, vm::PageType pageType=kDefault, int prot=3)  Line 389 + 0x100 bytes	C++
 	pyrogenesis.exe!da_alloc(DynArray * da=0x0074f840, unsigned int max_size=368902144)  Line 66 + 0x1b bytes	C++
 	pyrogenesis.exe!tex_encode(Tex * t=0x0074f8ac, const Path & extension={...}, DynArray * da=0x0074f840)  Line 761 + 0x11 bytes	C++
 	pyrogenesis.exe!tex_write(Tex * t=0x0074f8ac, const Path & filename={...})  Line 176 + 0x1d bytes	C++
 	pyrogenesis.exe!WriteBigScreenshot(const Path & extension={...}, int tiles=10)  Line 372 + 0x10 bytes	C++
 	pyrogenesis.exe!MainInputHandler(const SDL_Event_ * ev=0x0074fac8)  Line 151 + 0x1c bytes	C++
 	pyrogenesis.exe!in_dispatch_event(const SDL_Event_ * ev=0x0074fac8)  Line 60 + 0xa bytes	C++
 	pyrogenesis.exe!PumpEvents()  Line 193	C++
 	pyrogenesis.exe!Frame()  Line 356	C++
 	pyrogenesis.exe!RunGameOrAtlas(int argc=1, const char * * argv=0x00000002)  Line 507 + 0x5 bytes	C++
 	pyrogenesis.exe!main(int argc=1, char * * argv=0x00967a50)  Line 550 + 0xc bytes	C++
 	pyrogenesis.exe!wmain(int argc=1, wchar_t * * argv=0x02497b70)  Line 380 + 0xa bytes	C++
 	pyrogenesis.exe!__tmainCRTStartup()  Line 552 + 0x17 bytes	C
 	pyrogenesis.exe!CallStartupWithinTryBlock()  Line 397	C++
 	kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes	
 	ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes	
 	ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes	

The easy way to reproduce it (IMO) is playing a large map with AIs, set the simulation rate to 8.0 or so, and wait a while until the crash occurs when taking a big screenshot (takes about 25 minutes of game time).

Attachments (1)

largeaddressaware.patch (3.6 KB) - added by echotangoecho 3 years ago.
Allows the executable to use more than 2GB of ram on windows, and seems to fix this issue.

Download all attachments as: .zip

Change History (12)

comment:1 Changed 7 years ago by historic_bruno

Description: modified (diff)

comment:2 Changed 6 years ago by Markus

Keywords: windows added

comment:3 Changed 5 years ago by stanislas69

Keywords: unanswered added

comment:4 Changed 4 years ago by historic_bruno

#2532 was a duplicate.

comment:5 Changed 4 years ago by fabio

A workaround could be to disable big screenshots when on 32 bit and over a screen resolution that would exceed the available memory.

comment:6 Changed 4 years ago by AlThePhoenix

On my machine (Windows 7 64 Bit) I have never been able to create a big screenshot. If I try to take one in either Atlas or in a match, the game crashes instantly. It doesn't shut down completely though, I am still able to press Continue, after which the engine continues like it should. It should be noted that this even occurs on an empty tiny map.

Changed 3 years ago by echotangoecho

Attachment: largeaddressaware.patch added

Allows the executable to use more than 2GB of ram on windows, and seems to fix this issue.

comment:7 Changed 3 years ago by echotangoecho

Keywords: patch rfc added; unanswered removed
Milestone: BacklogAlpha 21
Summary: Big screenshot fails on win32 (Out of address space)[PATCH] Big screenshot fails on win32 (Out of address space)

comment:8 Changed 3 years ago by elexis

From 2016-08-08-QuakeNet?-#0ad-dev.log

20:14 < Philip> echotangoecho: Might need to be careful that large addresses don't break any pointer-tagging tricks in SpiderMonkey etc
20:15 < Philip> The big screenshot thing should be fixed by making it not allocate obscene amounts of (virtual) memory
20:15 < echotangoecho> I don't think they should break any of that
20:17 < echotangoecho> especially because I think the limitation of 2GB addressable memory doesn't exist for 32 bit linux builds?
20:19 < Philip> They'd break anything that tries to store its own data in the top bit of pointers (based on the assumption that non-kernel 32-bit pointers always have 0 in the top bit), and SM at least used to do some weird tricks with packing pointers into the NaN space of doubles, though I think maybe that was only really a problem with 64-bit pointers (which are hard to fit into 64-bit doubles alongside all the actual numbers)
20:20 < Philip> Hmm, true, it sounds like Linux defaults to ~1GB for kernel addresses
20:21 < echotangoecho> IMO, it seems like a pretty bad assumption to make (that the top-bit is unused)
20:22 < echotangoecho> so it seems unlikely to me
20:22 < echotangoecho> but that said, I'm no spidermonkey expert
20:26 < Philip> https://wingolog.org/archives/2011/05/18/value-representation-in-javascript-implementations has some related information
20:26 < Philip> and https://evilpie.github.io/sayrer-fatval-backup/cache.aspx.htm#post-485
20:27 < echotangoecho> that's not related, afaics
20:27 < echotangoecho> that only uses the unused lower bits
20:27 < echotangoecho> because it assumes aligned memory
20:28 < Philip> On x64_64 it relies on knowing that the top ~17 bits are always identical (i.e. it's essentially a 48-bit address space)
20:28 < echotangoecho> yes
20:28 < Philip> and (I think) that bit 47 is always 0 for non-kernel addresses
20:28 < Philip> so it's a similar concept :-)
20:28 < echotangoecho> but see "The old way" in your last link
20:28 < Philip> but doesn't really apply to 32-bit addresses
20:29 < echotangoecho> that seems to be about 32 bit adresses
20:29 < Philip> That's about 32-bit addresses in a decade-old version of SpiderMonkey
20:29 < echotangoecho> oh
20:31 < echotangoecho> I'm not sure whether I really understand "The new way", do they just store 32 bit pointers inside a 64 bit value?
20:32 < Philip> On 32-bit platforms, ~6 years ago, yes
20:32 < echotangoecho> do you know of any more recent docs?
20:32 < Philip> https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Internals#JavaScript_values maybe
20:33 < echotangoecho> On x86, ARM, and similar 32-bit platforms, we use what we call "nunboxing", in which non-double values are a 32-bit type tag and a 32-bit payload, which is normally either a pointer or a signed 32-bit integer
20:33 < echotangoecho> so it seems unlikely they make use of the highest bit for tagging
20:36 < Philip> Yeah, in practice it sounds like it's okay

comment:9 Changed 3 years ago by Itms

In 18704:

Whitespace fixes, patch by echotangoecho, refs #1619

comment:10 Changed 3 years ago by Itms

Keywords: windows patch rfc removed
Milestone: Alpha 21Backlog
Summary: [PATCH] Big screenshot fails on win32 (Out of address space)Big screenshot fails on win32 (Out of address space)

I committed the whitespace and created a ticket for the large-address-aware flag: #4190.

The big screenshot stupidness is still to be worked on.

comment:11 Changed 3 years ago by Itms

In 18711:

Add a linker flag on Windows allowing the game to be large address aware. This allows the game to use more than 2GB of RAM. https://msdn.microsoft.com/en-us/library/wz223b1z.aspx

However, this can mask memory issues, so this flag will not be enabled by default. The autobuilder will use it, so SVN playtesters and release users can benefit from it. Windows users compiling the game themselves will have the 2GB limitation.

Fixes #4190, refs #1619, #4191
Flag tested by Stan

Note: See TracTickets for help on using tickets.