Opened 17 years ago

Closed 18 months ago

#187 closed enhancement (wontfix)

Heap usage profiling

Reported by: Philip Taylor Owned by:
Priority: Nice to Have Milestone:
Component: Core engine Keywords:
Cc: Patch:

Description (last modified by phosit)

Currently the in-game profiler shows the number of memory allocations per frame in each profile node (at least when compiled with MSVC and the Debug CRT). It might be nice to also show the total amount of still-allocated memory that has ever been allocated under a particular node, to see which bits are responsible for using lots of memory. I expect that could be done without much pain by using _CrtSetAllocHook to catch allocations and frees: allocations can be added to the count in the current profile node, and some kind of map can store the allocation request number and the current node (I'm hoping you can remember nodes like that), and frees can look up the appropriate node to subtract the size from (because we don't want to count freed memory, and because it's not necessarily going to be freed in the same place as it was allocated). That'd be a little slow, so it should be optional and disabled by default.

Attachments (2)

profile-example.txt (29.0 KB ) - added by Markus 11 years ago.
profile log created with the patch (beginning of "Acropolis 1")
profiler-memtracker.patch (7.7 KB ) - added by Markus 11 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 by Andrew, 13 years ago

Milestone: BetaBacklog

comment:2 by Markus, 11 years ago

I extended the hooked malloc/.../free (USE_GLIBC_MALLOC_OVERRIDE) and build a global map to include the allocated memory address, the current profiler node and the memory size. The free can than call the matching node to subtract the size.

The first problem was the recursive call to allocate/free memory for the map. I build some guard, so only the first call gets proccessed. (We dont want to keep track of the memory actually used in the map.)

Another one was the StructuralReset invalidating all node pointers.

Mutex must be used to be safe with multiple threads.

Its neither very clean nor very fast. But it is usable. (As a patch; to make it build on other platforms some preprocessors need to be added.)

Last edited 11 years ago by Markus (previous) (diff)

comment:3 by Markus, 11 years ago

Keywords: patch added
Summary: Heap usage profiling[PATCH] Heap usage profiling

by Markus, 11 years ago

Attachment: profile-example.txt added

profile log created with the patch (beginning of "Acropolis 1")

by Markus, 11 years ago

Attachment: profiler-memtracker.patch added

comment:4 by Markus, 11 years ago

Keywords: review added

comment:5 by historic_bruno, 11 years ago

Milestone: BacklogAlpha 15

comment:6 by Jorma Rebane, 11 years ago

Hey, finally got some time to review your patch.

At first glance it looks like a really overkill method for doing a really simple thing. Can't you just add a "long malloc_bytes;" to CProfileNode and just increment/decrement that value as necessary?

It would be far more easier and faster, no?

comment:7 by Jorma Rebane, 11 years ago

Keywords: review removed

comment:8 by leper, 10 years ago

Milestone: Alpha 15Backlog

comment:9 by Stan, 9 years ago

Keywords: patch removed
Summary: [PATCH] Heap usage profilingHeap usage profiling

comment:10 by phosit, 18 months ago

Description: modified (diff)
Milestone: Backlog
Resolution: wontfix
Status: newclosed

The allocation profiler got removed in [26805]

Note: See TracTickets for help on using tickets.