Heya Masm32!
I have lurked round the forums for some time now, but I finally have something to contribute...
I've taken the liberty of fixing Ernie Murphy's Alloc procedure from the Masm32 library.
The last version wrongly allocated resources, since regardless of what amount of memory you
requested it would call the HeapCreate with some random value acquired from the stack pointer.
This can be easily observed by stepping into the notorious "call DWORD PTR[ ecx + 12 ]".
(http://www.anime42.com/TestAlloc.png)
The main reason why no one noticed this so far would have to be that the procedure
DOES actually allocate memory, but how much of it - no one can guess. The final result was
that the procedure was actually usable, but it would allocate random amounts of memory.
At first I thought this was a transition bug from 32 to 64 bit.. or from win98 to NT...
but I've tested this hypothesis on XP 32 & 64 bit, Vista and last but least Win98.
It's just a silly bug :)
I would appreciate if someone could double check this for me, and for the next
ever-so-wonderful MASM32 package!
Attached:
Alloc.asm <- my proposed solution.. I've swapped a few lines of code around, nothing fancy.
Test\Alloc.asm <- old Alloc.asm with an "equ" switch to test the error
Test\Test.asm <- old Alloc tester
Test\Free.asm <- clean code rules :)
Test\Build.bat <- builds the example
Thanks everyone!
[attachment deleted by admin]
I forgot I reduced the TEST_DD_BLOCK whilst taking the screenshot :)
So, if you want the program to crash (which is a good thing, since it proves the bug),
just enlarge the TEST_DD_BLOCK to something like
TEST_DD_BLOCK equ 1000000h ; size of dd array to allocate, around 64Mb
Cheers!
The reason I swapped the code around was because there is a miniature gain from the swap...
@@: mov eax, [ Alloc_pIMalloc ]
.IF eax ; check if we hold a valid pointer
;{
push [ cb ]
push esp
mov eax, [ eax ]
call DWORD PTR [ eax + 12 ]
ret
;}
.ENDIF
; if allocating for the 1st time, create pIMalloc
invoke CoGetMalloc, 1, ADDR Alloc_pIMalloc
or eax, eax
jz @B ; if no error, return to Alloc
xor eax, eax ; failed getting pIMalloc, return NULL pointer
ret
From what I gathered, it's always better to drop through the .IF branch, and this way,
the .IF->.ENDIF jump gets executed only once during Alloc_pIMalloc initialization.
Also, this way we already have the Alloc_pIMalloc dereferenced and loadded into eax,
ready for the next round of dereferencing... and another one... :)
---
or eax, eax
jz @B ( or jz Alloc if you prefer )
is because I don't know how to do it neatly via the .IF high level syntax...
If I wrote something like
.IF !eax
jmp @B
.ENDIF
It would get translated to
or eax, eax
jnz @F
jmp @B
@@:
which is a bit crude, I think.
Sorry for gibbering like an idiot, but you know how it is with first posts on a forum...
It's like a sugar hype! :D
LimoDriver,
Welcome to the forum. I have verified that there is a problem. In my tests the allocation size is not random, but fixed at 0012FFF0h (1245168). And your patch, performing an extra push before the call does seem to fix the problem, but I don't know enough about the interface, or COM in general, to have much confidence in it.
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
include \masm32\include\masm32rt.inc
_Alloc PROTO :DWORD
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
.data
ptrs dd 100 dup(0)
.code
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
start:
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
xor ebx, ebx
.WHILE ebx < 100
mov esi, ebx
shl esi, 16
invoke Alloc, esi
mov [ptrs+ebx*4], eax
print uhex$([ptrs+ebx*4]),9
print uhex$(esi),9
invoke crt__msize, [ptrs+ebx*4]
print uhex$(eax),13,10
inc ebx
.ENDW
xor ebx, ebx
.WHILE ebx < 100
invoke Free, [ptrs+ebx*4]
inc ebx
.ENDW
print chr$(13,10)
inkey "Press any key to continue..."
print chr$(13,10)
xor ebx, ebx
.WHILE ebx < 100
mov esi, ebx
shl esi, 16
invoke _Alloc, esi
mov [ptrs+ebx*4], eax
print uhex$([ptrs+ebx*4]),9
print uhex$(esi),9
invoke crt__msize, [ptrs+ebx*4]
print uhex$(eax),13,10
inc ebx
.ENDW
xor ebx, ebx
.WHILE ebx < 100
invoke Free, [ptrs+ebx*4]
inc ebx
.ENDW
inkey "Press any key to exit..."
exit
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
.data
Alloc_pIMalloc dd 0
.code
_Alloc proc public cb:DWORD
; -------------------------------------------------------------
; Alloc will allocate cb bytes
;
; The Alloc method allocates a memory block in essentially the same
; way that the C Library malloc function does.
;
; The initial contents of the returned memory block are undefined
; there is no guarantee that the block has been initialized, so you should
; initialize it in your code. The allocated block may be larger than cb bytes
; because of the space required for alignment and for maintenance information.
;
; If cb is zero, Alloc allocates a zero-length item and returns a valid
; pointer to that item. If there is insufficient memory available, Alloc
; returns NULL.
;
; Note Applications should always check the return value from this method,
; even when requesting small amounts of memory, because there is no guarantee
; the memory will be allocated
;
;
; EXAMPLE:
; invoke Alloc, 128 ; allocates 128 bytes
; ; pointer to memory in eax
;
; Uses: eax, ecx, edx.
;
; -------------------------------------------------------------
.IF !Alloc_pIMalloc ; check if we hold a valid pointer
invoke CoGetMalloc,1 , ADDR Alloc_pIMalloc
.IF eax
; failed getting pIMalloc
xor eax, eax ; NULL return pointer
ret
.ENDIF
.ENDIF
; now request the memory
push cb
mov ecx, Alloc_pIMalloc
mov ecx, [ecx]
push 0 ; <========================================
call DWORD PTR [ecx] + 12
ret
_Alloc endp
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
end start
Thank you very much for you time!
> In my tests the allocation size is not random, but fixed at 0012FFF0h (1245168).
This is quite possible, but only because you invoked the Alloc procedure from a fixed position in your code.
You see, the main problem with the current implementation is that it calls a function which accepts two parameters, the first one being ignored, and the latter one being the amount of memory to allocate. This is clearly visible by the RETN 8 at the end of the memory allocating procedure.
Since the original Alloc code only pushed >one< parameter, the procedure mistakenly grabs the current EBP pointer as the amount of memory - since the construction of the procedural frame pushes ebp via "the olde"
push ebp
mov ebp, esp
Since your stack frame pointer is fixed into whatever position you performed the call from, it will always be 0012FFF0h, but if you change the EBP pointer, it will also change.
The only thing that saves the day (from stack corruption) is the "leave" instruction inside the memory allocating procedure. It "fixes" the incorrect call by aligning the EBP/ESP at the end of the frame. It's also the reason why the bug remained undetected for so long, IMO.
Thanx again!
Hi Limo,
Welcome on board. Thanks for the effort on this procedure. I treat COM like the plague as I am more at home with Heap,Virtual or GlobalAlloc which always do what I need so I have never used it. Ernie has long retired so if you can produce a reliable method of fixing his old algo it would be most appreciated.
Thank you for the warm welcome.
In my opinion, there are two ways to skin the cat in question:
1) Just add an extra push and call it a day
As I said before, I've tested this on various windows systems, actually mostly anything except Windows ME.
Everything was smooth, and the memory allocated neatly fitted the requested amount.
2) Abandon the whole COM concept and just create a new set of Alloc and Free to allow beginners a simple
and efficient way of hanging to old concepts :)
For instance, a quick suggestion, written from the top of my head
Alloc proc public cb:DWORD
;{
invoke GetProcessHeap
invoke HeapAlloc, eax, 0, [ cb ]
ret
;}
endp
Free proc public pMem:DWORD
;{
invoke GetProcessHeap
invoke HeapFree, eax, 0, [ pMem ]
ret
;}
endp
The call is up to you.
I think it would still be nice to have the old fixed Alloc, since the above mentioned procedures are a bit boring :)
EDIT: ZOMG and they occupy more space in the executable!
Limo,
Probably the best approach is to preserve the technical type in Ernie's algos as the masm32 project has a number of simplified macros for memory allocation that are just easier to use than procedures of this type. Noting the case difference, the pair of fixed allocation macros are respectively "alloc" and free" based on GlobalAlloc() and there are a couple of others that use heap memory and OLE memory.
The main thing is to keep the algos compatible so they don't break existing code if someone has already used them in their code design. If you have the time to properly fix the algo I would be more than happy to update the algo in the library so it works correctly. What I need is the complete algo posted in a form that has been tested to show it works correctly.
QuoteThe main thing is to keep the algos compatible so they don't break existing code if someone has already used them in their code design.
Of course. The only code that will break after the library update will be code that allocates less memory then required for the sole application, because
it was defaulting to around 1Mb+, as the conversation with MichaelW showed. We aren't changing the interface, just removing the bug.
QuoteIf you have the time to properly fix the algo I would be more than happy to update the algo in the library so it works correctly.
What I need is the complete algo posted in a form that has been tested to show it works correctly.
This is the version that has been tested to show it works on all Windows systems:
; #########################################################################
; -------------------------------------------------------
; This procedure was written by Ernie Murphy 10/1/00
; -------------------------------------------------------
.386
.model flat, stdcall ; 32 bit memory model
option casemap : none ; case sensitive
PUBLIC Alloc_pIMalloc
include \masm32\include\ole32.inc
includelib \masm32\lib\ole32.lib
.data
Alloc_pIMalloc dd 0
.code
; #########################################################################
Alloc proc public cb : DWORD
; -------------------------------------------------------------
; Alloc will allocate cb bytes
;
; The Alloc method allocates a memory block in essentially the same
; way that the C Library malloc function does.
;
; The initial contents of the returned memory block are undefined
; there is no guarantee that the block has been initialized, so you should
; initialize it in your code. The allocated block may be larger than cb bytes
; because of the space required for alignment and for maintenance information.
;
; If cb is zero, Alloc allocates a zero-length item and returns a valid
; pointer to that item. If there is insufficient memory available, Alloc
; returns NULL.
;
; Note: Applications should always check the return value from this method,
; even when requesting small amounts of memory, because there is no guarantee
; the memory will be allocated
;
;
; EXAMPLE:
; invoke Alloc, 128 ; allocates 128 bytes
; ; returns pointer to memory in eax
;
; Uses: eax, ecx, edx.
; -------------------------------------------------------------
@@: mov eax, [ Alloc_pIMalloc ]
.IF eax ; check if we hold a valid pointer
;{
mov eax, [ eax ]
push [ cb ]
push 0h
call DWORD PTR [ eax + 12 ]
ret
;}
.ENDIF
; if allocating for the 1st time, create pIMalloc
invoke CoGetMalloc, 1, ADDR Alloc_pIMalloc
or eax, eax
jz @B ; if no error, return to Alloc
xor eax, eax ; failed getting pIMalloc, return NULL pointer
ret
Alloc endp
; #########################################################################
end
Despite the RETN 8 this is still a blind fix IMO. What is this other parameter, and why does it exist? Can it be any value, or are there circumstances where it must be a specific value? Looking at the methods for the IMalloc interface (http://msdn2.microsoft.com/en-us/library/ms678425.aspx) I noticed that some of the methods have two documented parameters, and suspected that this might be the reason for the extra parameter for the methods that have only one documented parameter. Testing the Realloc method, which has two documented parameters, I determined that I had to add a dummy push for it as well.
BTW, Free also needs to be fixed, and I think it should at least check for a valid pointer.
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
include \masm32\include\masm32rt.inc
_Alloc PROTO :DWORD
Realloc PROTO :DWORD,:DWORD
GetSize PROTO :DWORD
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
.data
ptrs dd 100 dup(0)
.code
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
start:
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
; So Free will work.
invoke Alloc, 0
invoke Free, eax
xor ebx, ebx
.WHILE ebx < 100
mov esi, ebx
shl esi, 16
invoke _Alloc, esi
mov [ptrs+ebx*4], eax
print uhex$([ptrs+ebx*4]),9
print uhex$(esi),9
invoke GetSize, [ptrs+ebx*4]
print uhex$(eax),13,10
inc ebx
.ENDW
xor ebx, ebx
.WHILE ebx < 100
invoke Free, [ptrs+ebx*4]
inc ebx
.ENDW
print chr$(13,10)
inkey "Press any key to continue..."
print chr$(13,10)
invoke _Alloc, 1000000h
mov edi, eax
print uhex$(edi),13,10
invoke GetSize, edi
print uhex$(eax),13,10
mov ebx, 1000000h
.WHILE ebx
invoke Realloc, edi, ebx
invoke GetSize, edi
print uhex$(eax),13,10
shr ebx, 1
.ENDW
invoke Free, edi
inkey "Press any key to exit..."
exit
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
.data
Alloc_pIMalloc dd 0
.code
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
_Alloc proc public cb : DWORD
; -------------------------------------------------------------
; Alloc will allocate cb bytes
;
; The Alloc method allocates a memory block in essentially the same
; way that the C Library malloc function does.
;
; The initial contents of the returned memory block are undefined
; there is no guarantee that the block has been initialized, so you should
; initialize it in your code. The allocated block may be larger than cb bytes
; because of the space required for alignment and for maintenance information.
;
; If cb is zero, Alloc allocates a zero-length item and returns a valid
; pointer to that item. If there is insufficient memory available, Alloc
; returns NULL.
;
; Note: Applications should always check the return value from this method,
; even when requesting small amounts of memory, because there is no guarantee
; the memory will be allocated
;
;
; EXAMPLE:
; invoke Alloc, 128 ; allocates 128 bytes
; ; returns pointer to memory in eax
;
; Uses: eax, ecx, edx.
; -------------------------------------------------------------
@@: mov eax, [ Alloc_pIMalloc ]
.IF eax ; check if we hold a valid pointer
;{
mov eax, [ eax ]
push [ cb ]
push 0h
call DWORD PTR [ eax + 12 ]
ret
;}
.ENDIF
; if allocating for the 1st time, create pIMalloc
invoke CoGetMalloc, 1, ADDR Alloc_pIMalloc
or eax, eax
jz @B ; if no error, return to Alloc
xor eax, eax ; failed getting pIMalloc, return NULL pointer
ret
_Alloc endp
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
Realloc proc pMem:DWORD,cb:DWORD
@@:
mov eax, [Alloc_pIMalloc]
.IF eax
mov eax, [eax]
push cb
push pMem
push 0
call DWORD PTR [eax+16]
ret
.ENDIF
invoke CoGetMalloc, 1, ADDR Alloc_pIMalloc
or eax, eax
jz @B
xor eax, eax
ret
Realloc endp
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
GetSize proc pMem:DWORD
@@:
mov eax, [Alloc_pIMalloc]
.IF eax
mov eax, [eax]
push pMem
push 0
call DWORD PTR [eax+24]
ret
.ENDIF
invoke CoGetMalloc, 1, ADDR Alloc_pIMalloc
or eax, eax
jz @B
xor eax, eax
ret
GetSize endp
; «««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««««
end start
Hi,
> Despite the RETN 8 this is still a blind fix IMO. What is this other parameter, and why does it exist? Can it be any value, or are there circumstances where it must be a specific value?
Theoretically it cannot be "any" value. With CoGetMalloc you get an IMalloc interface pointer, and this interface pointer then has to be the first parameter for any IMalloc method calls, among them IMalloc::Alloc. It's hidden in the C header files, but in MASM the parameter is not hidden of course.
the correct calling sequence would be:
invoke CoGetMalloc, 1, addr pMalloc
mov ecx, pMalloc
mov ecx, [ecx] ;get the vft
push dwBytes
push pMalloc
call [ecx+12] ;this is the 4. method of IMalloc (QueryInterface, AddRef, Release, Malloc)
However, IMO iit seems not a good idea to use the OLE heap for simple memory requests. Why not use just HeapAlloc?
QuoteBTW, Free also needs to be fixed, and I think it should at least check for a valid pointer.
I'm not sure why it needs to be fixed... the current version works all right.
It releases the memory, and it already does checks for zero pointer...
QuoteTheoretically it cannot be "any" value. With CoGetMalloc you get an IMalloc interface pointer, and this interface pointer then has to be the first parameter for any IMalloc method calls, among them IMalloc::Alloc.
Holy Zarquon, I think you're right! This would make perfect sense... And would fit tightly into the current implementation of Free.
It's just that the default implementation of IMalloc doesn't actually use that parameter... You can send it 0h in Allocs and Frees and it wouldn't give a wet slap.
So, the whatever the IMalloc call (currently working on Alloc, Free, Realloc), the first parameter should be [ Alloc_pIMalloc ].
This is my version of the IMalloc API.
The original CoGetMalloc was inside the Alloc function, so it wasn't available to Realloc,
GetSize or other IMalloc functions that will / may be implemented.
This version doesn't broadcast the Alloc_pIMalloc pointer like the last one, no need to do
externdef Alloc_pIMalloc : DWORD
anymore because IMalloc function is the only one that actually needs access to this pointer!
I've tried to produce documentation in a similar manner that Ernie Murphy's original code did:
; #########################################################################
.386
.model flat, stdcall ; 32 bit memory model
option casemap : none ; case sensitive
include \masm32\include\ole32.inc
includelib \masm32\lib\ole32.lib
; #########################################################################
.data
Alloc_pIMalloc dd 0
.code
; #########################################################################
IMalloc proc public
;{
;--------------------------------------------------------------------------
; IMalloc returns the interface pointer to the memory allocator.
; This interface pointer cannot be used from a remote process -
; each process must have its own allocator.
;
; If an error occured while fetching the interface pointer,
; IMalloc returns NULL.
;
; EXAMPLE:
; invoke IMalloc ; returns interface pointer in eax
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
lea ecx, [ Alloc_pIMalloc ]
mov eax, [ ecx ]
.IF eax
;{
ret
;}
.ENDIF
@@: invoke CoGetMalloc, 1, ecx
or eax, eax
jz IMalloc
;{
xor eax, eax
ret
;}
;}
IMalloc endp
end
; #########################################################################
.386
.model flat, stdcall ; 32 bit memory model
option casemap : none ; case sensitive
include \masm32\include\ole32.inc
includelib \masm32\lib\ole32.lib
; #########################################################################
IMalloc PROTO
.code
; #########################################################################
Alloc proc public cb : DWORD
;{
;--------------------------------------------------------------------------
; The Alloc function uses the default allocator to allocate a memory block
; in the same way that malloc does. The initial contents of the returned
; memory block are undefined – there is no guarantee that the block has
; been initialized. The allocated block may be larger than cb bytes because
; of the space required for alignment and for maintenance information.
;
; If cb is zero, Alloc allocates a zero-length item and returns a valid
; pointer to that item. If there is insufficient memory available, Alloc
; returns NULL.
;
; Applications should always check the return value from this method, even
; when requesting small amounts of memory, because there is no guarantee
; the memory will be allocated.
;
; EXAMPLE:
; invoke Alloc, 128 ; allocates 128 bytes, returns memory pointer in eax
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
invoke IMalloc
.IF eax
;{
push [ cb ]
push eax
mov eax, [ eax ]
call DWORD PTR [ eax + 12 ]
;}
.ENDIF
ret
Alloc endp
end
; #########################################################################
.386
.model flat, stdcall ; 32 bit memory model
option casemap : none ; case sensitive
include \masm32\include\ole32.inc
includelib \masm32\lib\ole32.lib
; #########################################################################
IMalloc PROTO
.code
; #########################################################################
Realloc proc public pv : DWORD, cb : DWORD
;{
;--------------------------------------------------------------------------
; The Realloc function changes the size of a memory block allocated by
; a previous call to Alloc or Realloc, in the same way that realloc does.
;
; The pv argument points to the beginning of the memory block.
; If pv is NULL, Realloc allocates a new memory block in the same way as
; the Alloc function. If pv is not NULL, it should be a pointer returned by
; a prior call to Alloc or Realloc.
;
; The cb argument specifies the size (in bytes) of the new block. The
; contents of the block are unchanged up to the shorter of the new and old
; sizes, although the new block can be in a different location. Because the
; new block can be in a different memory location, the pointer returned by
; Realloc is not guaranteed to be the pointer passed through the pv argument.
;
; If pv is not NULL and cb is zero, then the memory pointed to by pv is
; freed, with a return value of NULL.
;
; If pv is not NULL, cb is a valid size, and there was not enough memory
; available to expand the block to the given size, Realloc will return NULL,
; and the original block will remain unchanged.
;
; EXAMPLE:
; invoke Realloc, pv, 256 ; reallocates the memory at pv, pointer in eax
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
invoke IMalloc
.IF eax
;{
push [ cb ]
push [ pv ]
push eax
mov eax, [ eax ]
call DWORD PTR [ eax + 16 ]
;}
.ENDIF
ret
;}
Realloc endp
end
; #########################################################################
.386
.model flat, stdcall ; 32 bit memory model
option casemap : none ; case sensitive
include \masm32\include\ole32.inc
includelib \masm32\lib\ole32.lib
; #########################################################################
IMalloc PROTO
.code
; #########################################################################
GetSize proc public pv : DWORD
;{
;--------------------------------------------------------------------------
; The GetSize function returns the size of a memory block allocated by a
; previous call to Alloc or Realloc. If pv is a NULL pointer, the return
; value is -1.
;
; The size returned is the actual size of the allocation, which may be
; greater than the size requested when the allocation was made due to
; space required for alignment and for maintenance information.
;
; EXAMPLE:
; invoke GetSize, pv ; returns the memory block size (in bytes) in eax
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
invoke IMalloc
.IF eax
;{
push [ pv ]
push eax
mov eax, [ eax ]
call DWORD PTR [ eax + 24 ]
;}
.ENDIF
ret
;}
GetSize endp
end
; #########################################################################
.386
.model flat, stdcall ; 32 bit memory model
option casemap : none ; case sensitive
include \masm32\include\ole32.inc
includelib \masm32\lib\ole32.lib
; #########################################################################
IMalloc PROTO
.code
; #########################################################################
Free proc public pv : DWORD
;{
;--------------------------------------------------------------------------
; The Free function, using the default OLE allocator, frees a block of memory
; previously allocated through a call to the Alloc or Realloc function.
;
; The number of bytes freed equals the number of bytes that were originally
; allocated or reallocated. After the call, the memory block pointed to by
; pv is invalid and can no longer be used.
;
; The pv parameter can be NULL. If so, this method has no effect.
;
; EXAMPLE:
; invoke Free, pv ; frees the memory at pv, doesn't return a value (void)
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
invoke IMalloc
.IF eax
;{
push [ pv ]
push eax
mov eax, [ eax ]
call DWORD PTR [ eax + 20 ]
;}
.ENDIF
ret
;}
Free endp
end
And lastly, here is a little program demonstrating usage of the above API.
The following program will:
1) Allocate a block of 64Mb
2) Check the size
3) Fill the block with values
4) Reallocate the block to a new size of 128Mb
5) Check the new size
6) Fill the block with values
7) Free the memory
; #########################################################################
.386
.model flat, stdcall
option casemap : none ; case sensitive
; #########################################################################
IMalloc PROTO
Alloc PROTO :DWORD
Realloc PROTO :DWORD, :DWORD
GetSize PROTO :DWORD
Free PROTO :DWORD
.code
; #########################################################################
TEST_DD_BLOCK1 equ 4000000h ; size of dd array to allocate, around 64Mb
TEST_DD_BLOCK2 equ 8000000h ; size of dd array to reallocate, around 128Mb
start:
;{
int 3
invoke Alloc, TEST_DD_BLOCK1
mov ebx, eax
; ebx holds a pointer to 64Mb of uninitialized, allocated memory
invoke GetSize, ebx
cmp eax, TEST_DD_BLOCK1
; zero flag is set, showing that the size is truly 64Mb
mov eax, 12345678h
mov edi, ebx
mov ecx, TEST_DD_BLOCK1 / 4
cld
rep stosd
; all the memory has now been initialized to 12345678h
invoke Realloc, ebx, TEST_DD_BLOCK2
; the ebx pointer is no longer a valid one
; a new array has been allocated and holds the old values
; the rest of the array is initialised with zeroes
mov ebx, eax
; ebx now holds a pointer to 128Mb, first 64Mb being the old memory buffer
invoke GetSize, ebx
cmp eax, TEST_DD_BLOCK2
; size is a.o.k.
mov eax, 02468ACEh
mov edi, ebx
mov ecx, TEST_DD_BLOCK2 / 4
cld
rep stosd
; all the memory has now been initialized to 02468ACEh
invoke Free, ebx
; the ebx pointer is no longer a valid, memory has been released
xor eax, eax
ret
;}
end start
All the code posted here is attached in the IMalloc.zip attachment below.
There shouldn't be any more bugs, but if you do notice something wrong,
please give me a yell and I'll fix it pronto!
[attachment deleted by admin]
I tested before your last update, using my own test code, that includes calls to the Windows HeapWalk function, and AFAICT everything works correctly. I was somewhat surprised that the CRT _heapwalk function cannot enumerate these allocations, even though the _msize function can return the sizes.
[attachment deleted by admin]
QuoteI tested before your last update, using my own test code, that includes calls to the Windows HeapWalk function, and AFAICT everything works correctly.
Thanks, MichaelW. You've been most generous with your time, and you have restored my faith in console assembler.
This code will prove valuable for debugging my buggy OOP assemblies :)
QuoteI was somewhat surprised that the CRT _heapwalk function cannot enumerate these allocations, even though the _msize function can return the sizes.
CRT schmrt ... I never liked it very much... Even when I write stuff in C, I'd rather import from masm32lib then deal with MSVCRXX.dll version hell.
If you've got any suggestions about the above mentioned procedures, I'll be more than happy to change whatever looks ugly.
I'm not a COM with assembly specialist, so, i have a question...
Would it work?
OPTION PROLOGUE:NONE
OPTION EPILOGUE:NONE
align 16
Alloc proc public cb:DWORD
mov eax, Alloc_pIMalloc
test eax, eax
jnz haveptr
invoke CoGetMalloc,1 , ADDR Alloc_pIMalloc
test eax, eax
jnz saveptr
ret 8; if it jumps on eax != 0 eax must be zero here.
saveptr:
mov Alloc_pIMalloc, eax
haveptr:
mov ecx, [eax]
push [esp]; the return address
mov [esp+4], eax; pointer to the interface, a good pratice.
jmp [ecx]+12; let it do the rest
Alloc endp
OPTION PROLOGUE:PROLOGUEDEF
OPTION EPILOGUE:EPILOGUEDEF
Accepting suggestions!
; #########################################################################
; +=============================+=====++
; | Written by Ernie Murphy | '00 ||
; \ Tweaked by Limo Driver | '07 ||
; ^============================+=====++
; #########################################################################
.386
.model flat, stdcall ; 32 bit memory model
option casemap : none ; case sensitive
; #########################################################################
include \masm32\include\ole32.inc
includelib \masm32\lib\ole32.lib
; #########################################################################
IMalloc PROTO
Alloc PROTO :DWORD
Realloc PROTO :DWORD, :DWORD
GetSize PROTO :DWORD
Free PROTO :DWORD
DidAlloc PROTO :DWORD
HeapMinimize PROTO
; #########################################################################
IMallocProc MACRO procName, procOffset, paramCount
;{
if paramCount eq 0
;{
procName PROC public
;}
elseif paramCount eq 1
;{
procName PROC public p1 : DWORD
;}
elseif paramCount eq 2
;{
procName PROC public p1 : DWORD, p2 : DWORD
;}
endif
;{
invoke IMalloc
.IF eax
;{
if paramCount gt 1
;{
push DWORD PTR [ p2 ]
;}
endif
if paramCount gt 0
;{
push DWORD PTR [ p1 ]
;}
endif
push eax
mov eax, [ eax ]
call DWORD PTR [ eax + procOffset ]
;}
.ENDIF
ret
;}
procName ENDP
;}
ENDM
; #########################################################################
.data
Alloc_pIMalloc dd 0
.code
; #########################################################################
;--------------------------------------------------------------------------
; IMalloc returns the interface pointer to the memory allocator.
; This interface pointer cannot be used from a remote process -
; each process must have its own allocator.
;
; If an error occured while fetching the interface pointer,
; IMalloc returns NULL.
;
; EXAMPLE:
; invoke IMalloc ; returns interface pointer in eax
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
IMalloc proc public
;{
lea ecx, [ Alloc_pIMalloc ]
mov eax, [ ecx ]
.IF eax
;{
ret
;}
.ENDIF
@@: invoke CoGetMalloc, 1, ecx
or eax, eax
jz IMalloc
;{
xor eax, eax
ret
;}
;}
IMalloc endp
; #########################################################################
;--------------------------------------------------------------------------
; The Alloc function uses the default allocator to allocate a memory block
; in the same way that malloc does. The initial contents of the returned
; memory block are undefined – there is no guarantee that the block has
; been initialized. The allocated block may be larger than cb bytes because
; of the space required for alignment and for maintenance information.
;
; If cb is zero, Alloc allocates a zero-length item and returns a valid
; pointer to that item. If there is insufficient memory available, Alloc
; returns NULL.
;
; Applications should always check the return value from this method, even
; when requesting small amounts of memory, because there is no guarantee
; the memory will be allocated.
;
; EXAMPLE:
; invoke Alloc, 128 ; allocates 128 bytes, returns memory pointer in eax
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
IMallocProc Alloc, 12, 1
; #########################################################################
;--------------------------------------------------------------------------
; The Realloc function changes the size of a memory block allocated by
; a previous call to Alloc or Realloc, in the same way that realloc does.
;
; The pv argument points to the beginning of the memory block.
; If pv is NULL, Realloc allocates a new memory block in the same way as
; the Alloc function. If pv is not NULL, it should be a pointer returned by
; a prior call to Alloc or Realloc.
;
; The cb argument specifies the size (in bytes) of the new block. The
; contents of the block are unchanged up to the shorter of the new and old
; sizes, although the new block can be in a different location. Because the
; new block can be in a different memory location, the pointer returned by
; Realloc is not guaranteed to be the pointer passed through the pv argument.
;
; If pv is not NULL and cb is zero, then the memory pointed to by pv is
; freed, with a return value of NULL.
;
; If pv is not NULL, cb is a valid size, and there was not enough memory
; available to expand the block to the given size, Realloc will return NULL,
; and the original block will remain unchanged.
;
; EXAMPLE:
; invoke Realloc, pv, 256 ; reallocates the memory at pv, pointer in eax
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
IMallocProc Realloc, 16, 2
; #########################################################################
;--------------------------------------------------------------------------
; The Free function, using the default OLE allocator, frees a block of memory
; previously allocated through a call to the Alloc or Realloc function.
;
; The number of bytes freed equals the number of bytes that were originally
; allocated or reallocated. After the call, the memory block pointed to by
; pv is invalid and can no longer be used.
;
; The pv parameter can be NULL. If so, this method has no effect.
;
; EXAMPLE:
; invoke Free, pv ; frees the memory at pv, doesn't return a value (void)
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
IMallocProc Free, 20, 1
; #########################################################################
;--------------------------------------------------------------------------
; The GetSize function returns the size of a memory block allocated by a
; previous call to Alloc or Realloc. If pv is a NULL pointer, the return
; value is -1.
;
; The size returned is the actual size of the allocation, which may be
; greater than the size requested when the allocation was made due to
; space required for alignment and for maintenance information.
;
; EXAMPLE:
; invoke GetSize, pv ; returns the memory block size (in bytes) in eax
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
IMallocProc GetSize, 24, 1
; #########################################################################
;--------------------------------------------------------------------------
; The DidAlloc function is useful if a application is using multiple allocations and
; needs to know whether a previously allocated block of memory was allocated by a
; particular allocation.
;
; Return values can be 1, 0, or -1, signifying the following:
;
; 1 - The memory block was allocated by this IMalloc instance;
; 0 - The memory block was not allocated by this IMalloc instance;
; -1 - DidAlloc is unable to determine whether or not it allocated the memory block.
;
; If DidAlloc has been called with a NULL pointer, it will return -1.
;
; EXAMPLE:
; invoke DidAlloc, pv ; pv is a memory pointer, return value in eax
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
IMallocProc DidAlloc, 28, 1
; #########################################################################
;--------------------------------------------------------------------------
; The HeapMinimize function minimizes the heap as much as possible by
; releasing unused memory to the operating system, coalescing adjacent
; free blocks and committing free pages.
;
; Calling HeapMinimize is useful when an application has been running for
; some time and the heap may be fragmented.
;
; EXAMPLE:
; invoke HeapMinimize ; releases unused memory to the operating system
;
; Uses: eax, ecx, edx
;--------------------------------------------------------------------------
IMallocProc HeapMinimize, 32, 0
; #########################################################################
end
You have an error in you code...
Quote from: EduardoS on March 31, 2007, 11:57:18 PM
invoke CoGetMalloc,1 , ADDR Alloc_pIMalloc
test eax, eax
jz saveptr ; was jnz
Otherwise it *should* work, but it's highly fragile.
Stack frames are your friend, especially when evoking god knows who's API :P
This part just looks ugly:
if paramCount eq 0
;{
procName PROC public
;}
elseif paramCount eq 1
;{
procName PROC public p1 : DWORD
;}
elseif paramCount eq 2
;{
procName PROC public p1 : DWORD, p2 : DWORD
;}
endif
But since I never wrote macros before, I don't know how to set it up properly...
I tried something like
procName PROC public
if paramCount gt 0
p1 : DWORD
endif
if paramCount gt 1
, p2 : DWORD
endif
...and tons of other variations, but all failed... :(
Quote from: LimoDriver on April 01, 2007, 12:46:29 AM
You have an error in you code...
I was expecting receiving an address as result or null if fail, but this funcion returns a HRESULT and store the value in retval. It needs more changes than the jnz/jz.
Thank you for pointing the wrong line.
I hate COM.
The fix...
OPTION PROLOGUE:NONE
OPTION EPILOGUE:NONE
align 16
Alloc proc public cb:DWORD
mov eax, Alloc_pIMalloc
test eax, eax
jnz haveptr
invoke CoGetMalloc,1 , ADDR Alloc_pIMalloc
test eax, eax
jz saveptr
xor eax, eax; not zero...
mov Alloc_pIMalloc, eax; can i trust the previous function call will leave it zero?
ret 8
saveptr:
mov eax, Alloc_pIMalloc; its already saved, now get it...
haveptr:
mov ecx, [eax]
push [esp]; the return address
mov [esp+4], eax; pointer to the interface, a good pratice.
jmp [ecx]+12; let it do the rest
Alloc endp
OPTION PROLOGUE:PROLOGUEDEF
OPTION EPILOGUE:EPILOGUEDEF
It would appear that the method used here is more complex than necessary.
MSDN2: CoTaskMemAlloc (http://msdn2.microsoft.com/en-us/library/ms692727.aspx)
I disagree... The code I posted on the 1st of April :P, although a bit more complex (it's what... the macro expands to... 6 lines of assemler?), implements each of these functions; GetIMallocInterface, Alloc, Realloc, GetSize, Free, DidAlloc, HeapMinimize...
And you can even add more functions, like the increase/decrease reference count, which I didn't implement, but could be done in as little as just one line call to the macro!
The difference between these two implementations is that the program will have to retrieve the procedure address for CoTaskMemAlloc, CoTaskMemFree, and so on for each of these functions, while the implementation I posted only needs one: the retrieval of the CoGetMalloc interface pointer. This means less DLL imports and smaller binary size - while retaining the same speed and functionality.
To clarify, the CoGetMalloc method is more complex than necessary to implement the functions that Ernie Murphy originally implemented. Had he chosen to use CoTaskMemAlloc, etc, the bug probably would not have existed.
Quote
Had he chosen to use CoTaskMemAlloc, etc, the bug probably would not have existed.
Well, that is true, but think about it - if Ernie used CoTaskMemAlloc instead of going through the CoGetMalloc bit - what would be the point of placing that single call into the masm32 library :lol
QuoteWell, that is true, but think about it - if Ernie used CoTaskMemAlloc instead of going through the CoGetMalloc bit - what would be the point of placing that single call into the masm32 library
I don't understand. The functions are not in the masm32 library, but in the ole32.dll import library (or at least they are in the current version). Calling CoTaskMemAlloc, CoTaskMemFree, or CoTaskMemRealloc would involve no significant overhead beyond that required for CoGetMalloc. Perhaps I should add that my post was in no way meant as a criticism, what you have ended up with here is very good. I just happened to come across the CoTask_ functions, and thought it was interesting that a simpler method is available, and has been since Windows 95/NT 3.1.
Quote from: MichaelW
I don't understand.
Lemme try to rephrase - when I said "what would be the point of placing that single call into the masm32 library", I meant that there isn't a point to create a new set of Alloc/Free/Realloc and so on if you're just going to be using them to call another procedure which does the exact same thing, like
Alloc proc public cb : DWORD
;{
invoke CoTaskMemAlloc, cb
ret
;}
Alloc endp
Because of this, we can safely conclude that Ernie would not have created the above mentioned procedures, because they would be redundant - and wouldn't be worth the effort, as it wouldn't introduce anything new to the community.
Quote from: MichaelW
Calling CoTaskMemAlloc, CoTaskMemFree, or CoTaskMemRealloc would involve no significant overhead beyond that required for CoGetMalloc.
Of, course, I wouldn't call it "significant", but it is true - each and every one of these "CoTaskMemAlloc', 'CoTaskMemFree', or 'CoTaskMemRealloc' will need to be imported from ole32.dll, and thus the release application will need to have links to each and every of these in the imports section ... As I said this isn't 'significant' but it's always nicer to have the exact same functionality by importing just one of these functions - 'CoGetMalloc'.
BTW, although you said it isn't - I'm never afraid of criticism, especially if it's something I can improve on.
Thanx for the kind words.
Not contesting the drawback of having 2 more links int the import section but...
The IMalloc interface isn't there for some polymorphism?
example: When creating a ISPI Filter you can use a special function to alocate memory, the address this function is passed as an item of parameter pfc of the HttpFilterProc. A COM library can do something like this, receiving a IMalloc as pointer wich may point to the "normal" interface or to a custom one.
Sorry for my bad english...
Guys,
I have moved this topic to the Laboratory as it seems to more about algo design than a simple fix of a faulty algo. I would appreciate knowing what the end result will be as Limo has shown there is a problem with Ernie's original and I need a complete working replacement that has the same interface so it can be slotted back into the masm32 library without breaking existing code.
Quote from: EduardoS on April 07, 2007, 01:34:07 AM
The IMalloc interface isn't there for some polymorphism?
Mostly correct. It's there so some objects can provide their own memory allocation routines.
Since the Malloc function only uses the default allocator, CoTaskMemAlloc and CoGetMalloc->Alloc will give identical results.
I agree that there is little point in having a function which simply calls another function, though the benefit in this case is backwards compatibility. New code could entirely avoid the call to the MASM library and call CoTaskMemAlloc directly, but removing the function will break old code that uses it.
AFAICT, this is the most efficient replacement:
Malloc PROC cb:DWORD
jmp CoTaskMemAlloc
Malloc ENDP
Alternatively, remove the function and use
Malloc EQU CoTaskMemAlloc, though if someone's code expects the actual function to exist (to get a pointer to it) this one may not work.
Cheers,
Zooba :U
The added functions GetSize, HeapMinimize, etc are worthwhile IMO, and since AFAIK there are no CoTask_ equivalents, and assuming the CoGetMalloc interface is in fact now working correctly, I think it should be retained.