Opened 11 years ago
Closed 11 years ago
#1995 closed enhancement (fixed)
[PERFORMANCE] Memory allocation reduction
Reported by: | Jorma Rebane | Owned by: | Jorma Rebane |
---|---|---|---|
Priority: | Must Have | Milestone: | Alpha 15 |
Component: | Core engine | Keywords: | patch performance memory |
Cc: | Patch: |
Description (last modified by )
Using MTuner data from a short run of the game, we can drastically improve 0 A.D. core engine performance, by reducing the number of allocations in the detected hotspots.
For more information, check the Memory Performance thread on the forums: http://www.wildfiregames.com/forum/index.php?showtopic=17310&st=20#entry270448
Sub-tickets: modeldef.cpp: #2006 parser.cpp: #2005 guimanager.cpp: #2007 textrenderer.cpp: #2027 shaderdefines.cpp: #2029
<progress>, <memblock>, <total memops>, <stacktrace> (V), size 4, count 61320, modeldef.cpp:300 (V), size 8, count 61320, modeldef.cpp:301 (V), size 48, count 42028, parser.cpp:246 (V), size 36, count 40866, guimanager.cpp:273 & 265 (V), size 8, count 32749, modeldef.cpp:241 (V), size 32, count 27638, textrenderer.cpp:136 (V), size 8, count 21952, cgui.cpp:986 (V), size 16, count 21952, textrenderer.cpp:77 (V), size 16, count 21952, shaderdefines.cpp:135 (V), size 48, count 21952, cgui.cpp:954 (V), size 12, count 20688, modelrenderer.cpp:635 (V), size 112, count 17325, guirenderer.cpp:388 ( ), size 108, count 16896, patchrdata.cpp:173 (V), size 8, count 14852, shaderdefines.cpp:116 (x), size 48, count 14154, font.cpp:33 ( ), size 36, count 13230, patchrdata.cpp:1025 (V), size 48, count 12008, parser.cpp:981 ( ), size 24, count 9783, patchrdata.cpp:1025 (V), size 8, count 9645, modelrenderer.cpp:433 (V), size 8, count 8935, modelrenderer.cpp:698 ( ), size 48, count 8265, cgui.cpp:916 (V), size 48, count 8144, parser.cpp:933 ( ), size 93, count 7806, guitext.cpp:266 ( ), size 132, count 7806, cgui.cpp:689 ( ), size 132, count 7806, cgui.cpp:800 & 855 (V), size 12, count 7072, modelrenderer.cpp:636 (V), , , textrenderer.cpp:77 (V), , , textrenderer.cpp:179 ( ), , , patchrdata.cpp:1052 ( ), , , guitext.cpp:210 ( ), , , terraintextureentry.h:75 ( ), , , patchrdata.cpp:192 ( ), , , cgui.cpp:675 ( ), , , texturemanager.cpp:150 ( ), , , texturemanager.cpp:153 ( ), , , texturemanager.cpp:511 ( ), , , texturemanager.h:136 (V), , , parser.cpp:639 ( ), , , componentmanager.cpp:620
Attachments (1)
Change History (29)
comment:1 by , 11 years ago
Description: | modified (diff) |
---|
comment:2 by , 11 years ago
Description: | modified (diff) |
---|
comment:5 by , 11 years ago
Description: | modified (diff) |
---|
comment:6 by , 11 years ago
Description: | modified (diff) |
---|---|
Summary: | [Performance] Memory allocation reduction → [PERFORMANCE] Memory allocation reduction |
comment:7 by , 11 years ago
Description: | modified (diff) |
---|
comment:8 by , 11 years ago
Keywords: | patch review added |
---|---|
Milestone: | Backlog → Alpha 14 |
comment:10 by , 11 years ago
Description: | modified (diff) |
---|
CTextRenderer patch also fixes cgui.cpp:986
comment:11 by , 11 years ago
Description: | modified (diff) |
---|
comment:12 by , 11 years ago
Description: | modified (diff) |
---|
Added ShaderDefines fix. Check ticket #2029
comment:13 by , 11 years ago
Description: | modified (diff) |
---|
comment:14 by , 11 years ago
Description: | modified (diff) |
---|
comment:15 by , 11 years ago
Patch applied cleanly with svn patch. Compiled fine with Xcode for OS X (10.8.4) but for "graphics". I get the following errors:
-in ShaderManager.cpp, line 87, "Cannot pass object of non-POD type 'ShaderDefStr' through variadic method; call will abort at runtime"
-in ModelRenderer.cpp, there are several calls to "_alloca" (namely l501/506/510/580), my compiler expects "alloca"
Starting the game worked fine. I could load a map after recompiling Collada and clearing the cache, but I get:
"HWLightingModelRenderer.cpp(237): Assertion failed: "streamflags == (streamflags & (STREAM_POS | STREAM_UV0 | STREAM_UV1 | STREAM_NORMAL))" and a MEM_BAD_ACCESS error in the renderer. Any idea what might cause this? Mistake on my side?
comment:16 by , 11 years ago
Latest patch fixed the compiler errors, but I get the same crash, here's the stacktrace. There's an assertion failure in AssertPointerBounds() in HWLightingModelRenderer.cpp at line 284 first.
* thread #1: tid = 0x1d07, 0x0000000107185c9d GLEngine`gleRunVertexSubmitImmediate + 5021, stop reason = EXC_BAD_ACCESS (code=1, address=0x0) frame #0 : 0x0000000107185c9d GLEngine`gleRunVertexSubmitImmediate + 5021 frame #1 : 0x0000000107178d67 GLEngine`gleDrawArraysOrElements_ExecCore + 1079 frame #2 : 0x0000000107171b33 GLEngine`glDrawRangeElements_IMM_Exec + 350 frame #3 : 0x00007fff8eec359f libGL.dylib`glDrawRangeElements + 45 frame #4 : 0x000000010050a218 pyrogenesis`ShaderModelVertexRenderer::RenderModel () + 376 at HWLightingModelRenderer.cpp:301 frame #5 : 0x000000010050dd3c pyrogenesis`ShaderModelRenderer::Render(this=0x0000000101aa2e30, modifier=0x0000000101a78d80, context=0x0000000000000001, flags=1) + 3580 at ModelRenderer.cpp:634 frame #6 : 0x0000000100532009 pyrogenesis`CRendererInternals::CallModelRenderers (this=0x00000001015d4c00, context=0x00000001008a0438, flags=1) + 329 at Renderer.cpp:364 frame #7 : 0x000000010052a6ef pyrogenesis`CRenderer::RenderShadowMap (this=0x00000001015c8c00) + 815 at Renderer.cpp:931 frame #8 : 0x000000010052e115 pyrogenesis`CRenderer::RenderSubmissions (this=0x00000001015c8c00) + 1157 at Renderer.cpp:1506 frame #9 : 0x000000010052fd96 pyrogenesis`CRenderer::RenderScene (this=0x00000001015c8c00) + 134 at Renderer.cpp:1818 frame #10 : 0x0000000100492af6 pyrogenesis`CGameView::Render() + 38 at GameView.cpp:503 frame #11 : 0x000000010042a865 pyrogenesis`Render() + 421 at GameSetup.cpp:218 frame #12 : 0x0000000100424fbe pyrogenesis`CGame::ReallyStartGame () + 254 at Game.cpp:216 frame #13 : 0x0000000100002766 pyrogenesis`SDL_main [inlined] ProgressiveLoad () + 312 at main.cpp:251 frame #14 : 0x000000010000262e pyrogenesis`SDL_main [inlined] Frame() + 1035 at main.cpp:349 frame #15 : 0x0000000100002223 pyrogenesis`SDL_main [inlined] RunGameOrAtlas () + 459 at main.cpp:510 frame #16 : 0x0000000100002058 pyrogenesis`SDL_main() + 1288 at main.cpp:553 frame #17 : 0x000000010029eaa1 pyrogenesis`-[SDLMain applicationDidFinishLaunching:] + 97 frame #18 : 0x00007fff9006eeda CoreFoundation `_CFXNotificationPost + 2554 frame #19 : 0x00007fff8b6457b6 Foundation`-[NSNotificationCenter postNotificationName:object:userInfo :] + 64 frame #20 : 0x00007fff87e3052d AppKit `-[NSApplication _postDidFinishNotification] + 292 frame #21 : 0x00007fff87e30266 AppKit `-[NSApplication _sendFinishLaunchingNotification] + 216 frame #22 : 0x00007fff87e2d452 AppKit `-[NSApplication(NSAppleEventHandling) _handleAEOpenEvent:] + 566 frame #23 : 0x00007fff87e2d04c AppKit `-[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent :] + 351 frame #24 : 0x00007fff8b65f07b Foundation`-[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon :] + 308 frame #25 : 0x00007fff8b65eedd Foundation`_NSAppleEventManagerGenericHandler + 106 frame #26 : 0x00007fff8ff30078 AE`aeDispatchAppleEvent(AEDesc const*, AEDesc*, unsigned int, unsigned char*) + 307 frame #27 : 0x00007fff8ff2fed9 AE`dispatchEventAndSendReply(AEDesc const*, AEDesc*) + 37 frame #28 : 0x00007fff8ff2fd99 AE`aeProcessAppleEvent + 318 frame #29 : 0x00007fff89461709 HIToolbox`AEProcessAppleEvent + 100 frame #30 : 0x00007fff87e29836 AppKit `_DPSNextEvent + 1456 frame #31 : 0x00007fff87e28df2 AppKit `-[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue :] + 128 frame #32 : 0x00007fff87e201a3 AppKit `-[NSApplication run] + 517 frame #33 : 0x000000010029e884 pyrogenesis`CustomApplicationMain + 548 frame #34 : 0x000000010029ee60 pyrogenesis`main + 272 frame #35 : 0x00007fff86f3c7e1 libdyld.dylib`start + 1
comment:17 by , 11 years ago
For me too the last patch solved the compiler (G++) errors: http://pastebin.com/QrGxmZjH
But game crashed at the end of map loading: http://pastebin.com/JiawfMKN
(both pastebin link available 1 week).
comment:18 by , 11 years ago
RedFox: Great work so far! But please don't commit this without thorough testing by many other parties. It's still considered WIP/unstable until all developers have tested this on their machines (compiles and plays multiplayer game through without issues).
comment:19 by , 11 years ago
k776: That is only logical, k776. We will be doing a lot of testing on this before it is committed. :)
comment:20 by , 11 years ago
When running the tests I get this:
./test Running 277 tests.......................................................................................utf8.cpp(84): Function call failed: return value was -100703 (Invalid UTF-8 sequence) Function call failed: return value was -100703 (Invalid UTF-8 sequence) Location: utf8.cpp:84 (RaiseError) Call stack: (0x8f470a) ./test() [0x8f470a] (0x8a8201) ./test() [0x8a8201] (0x8a83fd) ./test() [0x8a83fd] (0x8a8dbb) ./test() [0x8a8dbb] (0x8a6d41) ./test() [0x8a6d41] (0x67810c) ./test() [0x67810c] (0x49bb87) ./test() [0x49bb87] (0x4329a4) ./test() [0x4329a4] (0x43b748) ./test() [0x43b748] (0x41c7cd) ./test() [0x41c7cd] (0x7f456d58fea5) /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f456d58fea5] (0x42d2e5) ./test() [0x42d2e5] errno = 0 (No error reported here) OS error = ?
And when starting the game it displays weird graphic fragments (nothing of the main menu is visible except the cursor).
When starting in autostart mode to avoid the buggy main-menu I get these assertion failures:
First:
Assertion failed: "streamflags == (streamflags & (STREAM_POS | STREAM_UV0 | STREAM_UV1 | STREAM_NORMAL))" Location: HWLightingModelRenderer.cpp:237 (BeginPass)
If I click continue:
Assertion failed: "(m_StreamFlags & ~m_ValidStreams) == 0" Location: ShaderProgram.cpp:896 (AssertPointersBound)
I was using this commands:
./pyrogenesis -quickstart -autostart="Oasis 04" -autostart-ai=1:qbot -autostart-ai=2:qbot -autostart-ai=3:qbot -autostart-ai=4:qbot
EDIT: I figured out that the main menu displays correctly when I set preferglsl = "flase" and gentangents = "false" in local.cfg.
comment:21 by , 11 years ago
Milestone: | Alpha 14 → Alpha 15 |
---|
comment:22 by , 11 years ago
Keywords: | patch, review, performance, memory → patch review performance memory |
---|
With the latest patch there is a strange bug in Atlas. I placed an Iberian fortress (left) but as soon as the next placement preview appears (right), the textures become corrupted:
I can keep placing messed up looking buildings, and only after the preview entity is no longer visible do they return to normal. Sometimes the bug is more subtle, the texture appears darker than normal. It happens with other buildings too.
(Reproducible on my system with all graphics options enabled)
comment:23 by , 11 years ago
Got this when running the game. It then failed to draw anything on the screen.
ERROR: Failed to compile shader 'shaders/glsl/gui_text.vs': ERROR: 0:2: '' : #version must occur before any other statement in the program ERROR: Failed to compile shader 'shaders/glsl/gui_basic.vs': ERROR: 0:2: '' : #version must occur before any other statement in the program ERROR: Failed to compile shader 'shaders/glsl/gui_solid.vs': ERROR: 0:2: '' : #version must occur before any other statement in the program
Edit: Fixed. historic_bruno pointed out that both patches needed to be applied. I didn't apply the GLSL one, thought it was optional.
comment:24 by , 11 years ago
Here is the log I have under Ubuntu 13.04 64bit: http://pastebin.com/KiZe59q5
A pyrogenesis bin is generated but crash directly.
follow-up: 26 comment:25 by , 11 years ago
You need to apply the Glsl patch too, otherwise it crashes on startup.
comment:26 by , 11 years ago
Replying to wraitii:
You need to apply the Glsl patch too, otherwise it crashes on startup.
I applied the megapatch on /source and the glsl on /binaries
Maybe I dit it wrong. I clean all and test again.
[edition] So I cleaned all and compiled again (last svn version at the moment: 13719).
glsl patch apply without any errors.
megapatch apply with one propriety error: http://pastebin.com/cCTMScz9
Make logs are a bit cleaner: http://pastebin.com/YTBVuxGE
But the game still crash. Here is the error:
ERROR: CCacheLoader failed to find archived or source file for: "shaders/fixed:gui_text.xml"</p> ERROR: Failed to load shader 'fixed:gui_text'</p>
[Edit] I did not have "preferglsl=true". Now that I have it, it does not crash, but I have a strange picture: http://img15.hostingpics.net/pics/389942screenshot0010.png
Maybe a problem with my graphic card.
comment:27 by , 11 years ago
Overwriting config settings in local.cfg seems to be broken with the patch. It works with the same unmodified revision.
EDIT: That wasn't quite accurate. Setting Windowed to true in local.cfg works, but setting hotkey.fps.toggle = "AltGr+F" or hotkey.camera.reset = "P" doesn't work.
by , 11 years ago
Attachment: | redfox_megapatch.zip added |
---|
Latest megapatch update. 1 known bugs: 1) zip archiver fails?
comment:28 by , 11 years ago
Keywords: | review removed |
---|---|
Resolution: | → fixed |
Status: | new → closed |
An attempt at separating megapatch has begun here: http://git.wildfiregames.com/gitweb/?p=0ad.git;a=shortlog;h=refs/heads/projects/philip/megapatch-split
Any useful parts can be pulled out, reviewed, and if necessary new tickets created for them. Most of this ticket is no longer applicable.
CModelDef fixes uploaded. This patch addresses two problems: 1) CModelDef loading performance
2) Complex dynamic objects in structs
The result: much better memory performance and usage for model loading.
In order for this to work, I had to change the version of the PMD (binary model cache) format to 4 and change the way models are saved to PMD. This may invalidate any existing model cache. Backwards compatibility for versions 1, 2, 3 is there.