The MASM Forum Archive 2004 to 2012

General Forums => The Campus => Topic started by: Biterider on April 07, 2005, 12:32:14 PM

Title: Memory leakage tool
Post by: Biterider on April 07, 2005, 12:32:14 PM
Hi
A week ago I saw a demo of a tool that helps to prevent memory leakage. I was impressed to see how it works.  :U
You have to compile a module with your sources and run the app. When the app ends, a summary is shown with all the lines in the code where memory was allocated and not released.

The idea behind the approach of this tool was very interesting and I'm interested to know if someone has done something in this direction implemented for MASM?

Regards,

Biterider
Title: Re: Memory leakage tool
Post by: pbrennick on April 07, 2005, 02:13:06 PM
Can you post the link to the demo?

Paul
Title: Re: Memory leakage tool
Post by: sluggy on April 07, 2005, 02:20:25 PM
NuMega / Compuware have had a tool that does this for a long time now, it integrates into Visual Studio, very effective.
Title: Re: Memory leakage tool
Post by: Biterider on April 07, 2005, 02:22:05 PM
http://www.dnjonline.com/articles/tools/iss16_tools_memory.html
Title: Re: Memory leakage tool
Post by: hutch-- on April 07, 2005, 03:32:35 PM
I am definitely "Olde Skewl" when it come to code design that uses memory. Applying the law of gravity, what goes up must come down so when you allocate memory, you also design where it is deallocated and further, you test it to make sure it works. If you design code this way you don't get leaks, if you slop memory allocation around all over the place, it is nearly impossible to clean it up after and while tools of this type help, its no substitute for designing the allocation strategy properly in the first place.
Title: Re: Memory leakage tool
Post by: Biterider on April 07, 2005, 03:51:28 PM
Hi Hutch
You are right and that's what the coders of this tool claim.
Never the less, Murphy is always present and nobody can say that he has written and complex application with 100% certainty that all memory issues were done properly. Putting things in this light, I think that such a tool is a must have to check serious applications that will run on customers machines.

Biterider
Title: Re: Memory leakage tool
Post by: thomasantony on April 07, 2005, 04:25:09 PM
Memory management is the main roadblock in my OS development.  I have been trying for some time but got bored.

Thomas :U
Title: Re: Memory leakage tool
Post by: pbrennick on April 07, 2005, 04:49:04 PM
Hi All,
Hutch is, of course, correct when it comes to your own development strategies.  But, if you regularly download and test other people's code on your machine, you better have a tool such as this in your arsenal unless you want to spend many hours examining the code belonging to others to determine if memory management is an issue.  *Or cold boot very often*

Also, as far as Visual Studio tools are concerned, all I can say is anyone who can afford these pricey packages is fortunate, indeed.  I live on a fixed income and that would certainly *break* it!

Paul
Title: Re: Memory leakage tool
Post by: rags on April 07, 2005, 05:06:29 PM
Paul,
I hear you, my meager "allowance" dont allow for these nice things. :lol
Title: Re: Memory leakage tool
Post by: Biterider on April 07, 2005, 06:35:00 PM
Hi all
I know that this tool is expensive and wouldn't it be nice if we write it by ourselves?  :8)

Possibly, a strategy to implement such a tool for MASM looks like this:

1. Hook all APIs that allocate and deallocate memory.
2. Keep tracking the lifecycle of the returned handles and pointers and were they were called.
3. Hook the ExitProcess API.
4. Show the results when ExitProcess was called.

So far, nothing too complicated, but the problem I face is to get and show the code where memory was allocated without freeing it. I can get the memory offset but it doesn't give me any feedback of the source code line. Even if I use a debugger, I don't know how to get the correct source code line. Does somebody have an idea?

Regards,

Biterider
Title: Re: Memory leakage tool
Post by: hitchhikr on April 07, 2005, 08:28:05 PM
I'm working on such stuff right now.
Title: Re: Memory leakage tool
Post by: Biterider on April 07, 2005, 08:49:22 PM
Hi Hichhikr
What approach are you following? How would you solve the source code addressing issue?

Regards,

Biterider
Title: Re: Memory leakage tool
Post by: hitchhikr on April 07, 2005, 09:46:51 PM
It's not implemented yet so there might be some flaws in the logic.
Basically all i may do is to patch the known memory allocation functions in the import table of the file (after the calling so i can store the returned value), retrieve the calling address from the stack and walk thru it until the first valid call is found (that is inside a relevant sourcecode + line number) and store this entry somewhere.
An entry is discarded if the relevant memory functions is triggered (if the passed parameter to the freeing function is found in the list then it is removed, depending of the kind of memory allocation, of course).
All stuff still remaining in the list may eventually be freed and reported somewhere.
I haven't considered other handles yet.

You may also want to take a look at this:
http://research.microsoft.com/sn/detours/
Title: Re: Memory leakage tool
Post by: Biterider on April 09, 2005, 06:30:44 PM
Hi all
I have done the first step for the MemLeak tool, that is the API hooking of 3 procs I wanted to monitor: HeapAlloc, HeapFree and HeapReAlloc. Already the first tests showed me a programming error I had done with the memory allocation!

I created an object I called IAT32Hook that simply replaces the entry of a given API in the "Import Address Table" with the address of my own proc. By this way I can keep a count of this API (on successful return) and compare, i.e. the count of the allocation against the freeing of heap memory. If both counts are equal all is OK, but if not, somebody has forgotten to release a memory block somewhere in the code...

I'm thinking to do the same with other APIs that always go in pairs like BeginPaint and EndPaint.

At this point, the code I posted here doesn't keep track of handles, pointers or memory offsets. It simply counts. Of course, more sophisticated strategies can be thought, i.e. to check if a returned memory handle was freed. If not, the offset of the memory allocation can be shown. This info could come from the return address of the call to my API substitute procedure. The problem here is to relate this info with the source code to show this place to the programmer. I'm not familiar with the COFF information, but I guess that it could be a way to achieve it. Some help on this point will be appreciated!  :8)

This tool is a free open source project. I'm using ObjAsm32 to test it, but it can be used on other asm development platforms, too.

Regards,

Biterider


[attachment deleted by admin]
Title: Re: Memory leakage tool
Post by: Eóin on April 09, 2005, 06:48:53 PM
MemProof (http://www.automatedqa.com/downloads/memproof/) is free and does something like that.
Title: Re: Memory leakage tool
Post by: Brett Kuntz on April 09, 2005, 07:32:16 PM
I was under the impression windows unallocated all memory when an application exited anyways? I mean if you make a simple program allocated 512mb's of ram, run it, ctrl-alt-del and check your memory, you'll notice it went down by 512mb's. Exit the app (without deallocating) and it'll go away and you'll have your 512mb's back.
Title: Re: Memory leakage tool
Post by: pbrennick on April 09, 2005, 09:13:31 PM
Biterider,
Memleak has problems.  If a debugger is not found by the app, when the app closes, it leaves a messagebox stating that error running and it will not go away.  Every time you try to close it, it reopens itself.

BTW:  kuntor, this error proves your assumption is incorrect for 'all' cases.  Windows will not force down a process left running, as is happening here.  Download his app and try it.

Paul
Title: Re: Memory leakage tool
Post by: Biterider on April 09, 2005, 09:31:34 PM
Hi pbrennick
You have to install the ObjAsm32 package where the debug window is present to display the output messages. As said before, the MemLeak app is only to show the first step and to see if it is possible. If you want to work without the debug window, strip away the DbgXXX macros and redirect the output to the console or better to a MessageBox. My tip is to examine carefully the IMT32Hook object.  :8)
Regards,
Biterider
Title: Re: Memory leakage tool
Post by: pbrennick on April 09, 2005, 11:07:55 PM
Actually, I have installed the package.  And it is installed in the default location (except it is on the d: drive.  ObjAsm32 asks where masm32 is installed and from there I just let it do its thing.  I am testing your executable, not a recompilation so the problem is there.

Paul
Title: Re: Memory leakage tool
Post by: pbrennick on April 09, 2005, 11:16:13 PM
bTW:  Even if the package is not installed your app should check for that and not generate endless messageboxes.

Paul
Title: Re: Memory leakage tool
Post by: Biterider on April 10, 2005, 05:25:43 AM
Hi pbrennick
A fast solution for this problem is to start DebugCenter.exe and the MemLeak.exe.
Can you check if in your system the environment variables OA32_Path and Masm32_Path are defined?
I think that the problem is that the path to the DebugCenter is statically linked into the MemLeak.exe. I'll see how to solve it definitively...  :wink
Regards,
Biterider
Title: Re: Memory leakage tool
Post by: Biterider on April 10, 2005, 07:31:59 AM
Hi pbrennick
I have made some changes in the Debug routines to solve the problem. Now the DebugCenter path is determined at runtime based on the OA32_PATH environmental variable. If the .exe is not found, an alert messagebox is shown only once...  :toothy
If it works, I'll upload the changed sources.

Regards,
Biterider


[attachment deleted by admin]
Title: Re: Memory leakage tool
Post by: pbrennick on April 10, 2005, 12:52:21 PM
Yeah, there are no environment variables set on my machine.  I used to set them but I have multiple programming languages installed and it became a real problem at times and now I just wont do it, even if they appear to be unique.

Paul